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ABBREVIATIONS AND SYMBOLS 


ABBREVIATIONS : 
AA 

AA-01 

ACC 

Agromet 

AI 

Biowindow 


Biostage 
or biophase 


Blind site 


Accuracy Assessment. 

LACIE Phase II AA Report for February 1976. 
adjustable crop calendar, 
agricultural/meteorological . 
analyst-interpreter . 

biological window — a Landsat data acquisition 
period that is related to the biostages of 
wheat development. The LACIE approach is 
based on the judgment that wheat can be sepa- 
rated adequately from other crops by analysis 
of up to four acquisitions of Landsat data 
during the growing season. The biowindow 
may be updated if there is a significant lag 
or advancement in the current crop calendar. 
The sequence chosen includes acquisitions 
during the following biowindows: 

1. Crop establishment — from 50 percent 
tillering to 50 percent jointing (bio- 
stage 2.3 to 3.0). 

2. Green — from 50 percent jointing to 

50 percent heading (biostage 3.1 to 4.0). 

3. Heading — from 50 percent heading to 
50 percent soft dough (biostage 4.1 to 
5.0) . 

4. Mature — from 50 percent soft dough to 

50 percent harvest (biostage 5.1 to 6.0). 

biological stage, biological phase — the spe- 
cific stage of development of a crop which 
can be recognized by a major change in plant 
structure; i.e., emergence after germination, 
jointing, heading, soft dough, ripening, and 
harvest, which are represented by integers on 
the Robertson Biometeorological Time Scale. 

a LACIE sample segment chosen at random after 
normal analysis; used for testing classifica- 
tion performance. 
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BMTS 


CAMS 

CAS 

CCEA 


Classification 


Classification 

error 


CMR 

COM 

CRD 


Crop calendar 


Crop calendar 
adjustment 


CUR 

CV 

DAPTS 


Group 2 segment 


Biometeorological Time Scale. 

Classification and Mensuration Subsystem. 

Crop Assessment Subsystem. 

Center for Climatological and Environmental 
Assessment — an organization of the National 
Oceanic and Atmospheric Administration, 

Columbia, Missouri. 

in computer-aided analysis of remotely sensed 
data, the process of assigning data points to 
specified classes by a testing process in 
which the spectral properties of each unknown 
data point are compared with spectral proper- 
ties typical of the subject being classified. 

a measure of the degree to which the LACIE 
CAMS overestimates or underestimates the wheat 
acreage in one or more LACIE samples. 

CAS Monthly Report. 

classification on microfilm. 

Crop Reporting District — a geographical area 
used by the U.S. Department of Agriculture 
for the collection and reporting of agricul- 
tural information; each district consists 
of several counties. 

a calendar depicting the biostages of the 
major crop types within a specified region 
during a calendar year. 

an adjustment made to the normal crop calen- 
dar on the basis of current meteorological 
data. 

CAS Unscheduled Report. 

coefficient of variation (estimate of the stand- 
ard deviation divided by the estimate of the mean) . 

Data Acquisition, Preprocessing, and Trans- 
mission Subsystem. 

LACIE segment in a county that historically 
produces small quantities of wheat/small 
grains; samples are allocated with proba- 
bility proportional to size. 
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Group 3 segment 


IE 

IMR 

ITS 

JSC 

LACIE 
Lands at 

LEC 

LPDL 

LPP 

MSS 

Multitemporal 

analysis 

NASA 

NOAA 

90-90 criterion 


LACIE segment in a county that historically 
produces very small quantities of wheat/small 
grains; estimates are based on the changes in 
acreage of group 1 and 2 segments from year 
to year. 

Information Evaluation. 

IE Monthly .Report. 

intensive test site — a LACIE segment in the 
United States or Canada on which detailed 
crop information is collected by using ground 
and airborne equipment. 

Lyndon B. Johnson Space Center of NASA, 

Houston, Texas. 

Large Area Crop Inventory Experiment. 

Land Satellite — formerly called ERTS (Earth 
Resources Technology Satellite) ; operates in 
a circular. Sun-synchronous, near-polar orbit 
of the Earth at an altitude of approximately 
915 kilometers; orbits the Earth about 14 
times a day and views the same scene approxi- 
mately every 18 days. 

Lockheed Electronics Company, Inc. 

LACIE Physical Data Library. 

LACIE Performance Predictor. 

Multispectral Scanner System or multispectral 
scanner — the remote sensing instrument on 
Landsat that measures reflected sunlight in 
various spectral bands or wavelengths. 

analysis of data sets over the same area 
acquired at different times during the grow- 
ing season. 

National Aeronautics and Space Administration. 

National Oceanic and Atmospheric Administration. 

criterion that the LACIE U.S. Great Plains 
production estimate be 90 percent accurate, 
at harvest, 90 percent of the time (in com- 
parison with the true value) . 
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PCC 


PFC 

PPS 

RTEB 


probability of correct classification, 
production film converter, 
probability propor-feional to size. 
Research, Test, and Evaluation Branch. 


Sample segment 


Sampling error 


US DA 

USDA/ASCS 


a 5- by 6-nautical~mile area selected by 
stratified random sampling; information is 
recorded by the MSS and transformed into 
computer-compatible tapes and film products. 

a measure of the degree to which the estimated 
wheat acreage in the LAC IE sample segments 
does not represent the wheat acreage contained 
in the stratum being sampled. 

U.S. Department of Agriculture. 

USDA Agricultural Stabilization and Conser- 
vation Service. 


USDA/FAS USDA Foreign Agricultural Service. 

USDA/SRS USDA Statistical Reporting Service. 


U.S. Great Plains an area encompassing the nine states of 

Colorado, Kansas, Minnesota, Montana, 

Nebraska, North and South Dakota, Oklahoma, 
and Texas; it is divided geographically into 
(1) the U.S. southern Great Plains, which 
includes Colorado, Kansas, Nebraska, 

Oklahoma, and Texas, and (2) the U.S. northern 
Great Plains, which includes Minnesota, 
Montana, and North and South Dakota. 

WMO World Meteorological Organization. 

YES Yield Estimation Subsystem. 


SYMBOLS: 

P wheat/small-grain proportion estimate. 


P 


W 


proportion of wheat harvested. 


a 


standard deviation. 


CV(W) 


CV for production estimate. 
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Proportion of wheat/small grains based on 
identification of each field in the blind 
site or ITS by USDA/ASCS personnel . 



1. INTRODUCTION 


The Large Area Crop Inventory Experiment (LACIE) is an inter- 
agency endeavor of the National Aeronautics and Space Administra- 
tion (NASA) , the National Oceanic and Atmospheric Administration 
(NOAA) , and the U.S. Department of Agriculture (USDA) . Its pur- 
poses are to demonstrate the economical importance of utilizing 
satellite remotely sensed data from the Land Satellite (Landsat) 
for agricultural applications; to test the capability of a system 
utilizing remote sensing in conjunction with climatological, 
meteorological, and conventional data to produce timely esti- 

y 

mates of the production of a major world crop prior to harvest; 
and to validate the technology and procedures for such a system. 

In accordance with the objectives of the LACIE, the Accuracy 
Assessment (AA) effort is designed to check the accuracy of the 
products from the experimental operations throughout the growing 
season and thereby determine if the procedures used are suffi- 
cient to accomplish the above objectives. The LACIE AA Team 
(appendix A) is responsible for the technical direction of the 
AA program. 

The purpose of this LACIE Phase II AA Plan is to provide the 
specific guidelines for evaluating the accuracy of the Phase II 
procedures and estimates of wheat/ small-grain acreage, yield, 
and production and thereby determine if the LACIE is satisfying 
the requirement that its production estimates be 90 percent 
accurate, at harvest, 90 percent of the time when compared with 
the true value of harvested wheat/ small grains (the 90-90 
criterion) . 

This document was prepared by Lockheed Electronics Company, Inc. 
(LEC) , Aerospace Systems Division, Houston, Texas, under contract 


1-1 



NAS 9-12200 for the Earth Observations Division, Science and 
Applications Directorate, Lyndon B. Johnson Space Center (JSC) 
Of NASA. 
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2 . BACKGROUND 


In accomplishing its objectives , the LAC IE uses Earth Resources 
Program technology in conjunction with meteorological and con- 
ventional agricultural information to examine three global crop 
seasons, each of which is designated as a LACIE phase. 

Phase I, which began in January 1975, was devoted primarily to 
identifying and estimating wheat acreage in the states of 
Colorado, Kansas, Minnesota, Montana, Nebraska, North and South 
Dakota, Oklahoma, and Texas (the U.S. Great Plains). Recognition 
analyses were conducted in other selected areas, and yield model 
development and yield feasibility determinations were made over 
selected regions in the United States. Data from the USDA Sta- 
tistical Reporting Service (USDA/SRS) were compared with LACIE 
estimates to determine the accuracy of LACIE performance. 

Phase I AA activities were initiated in July 1975, and tests for 
wheat acreage accuracy were conducted using segments for which 
ground-truth data were available. Initially statistical tests 
and comparisons of LACIE estimates with ground truth were made 
using data from 27 intensive test sites (ITS's) in eight states 
and 2 Canadian ITS’s; then, to test a greater number of acquisi- 
tions in a more concentrated area, ground truth was gathered 
from 30 LACIE operational segments (blind sites) m two states 
and withheld from the analyst-interpreters (AI’s) until process- 
ing was completed. The Classification and Mensuration Subsystem 
(CAMS) processed the data from the blind sites, and the AA Team 
compared the results of the various sampling and classification 
procedures used. Approximately 340 special analyses were conducted 
to support Phase I AA. 

In Phase II, which begins in October 1975, emphasis remains on 
the U.S. Great Plains but analysis will be extended also to 
include ITS’s in Canada. 
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3. OBJECTIVES AND SCOPE 


3.1 OBJECTIVES 

The requirements for AA are established, to evaluate and improve 
LACIE procedures and thereby to support the primary objectives 
of the LACIE as stated in section 1. Phase II AA will continue 
to test and evaluate LACIE estimates of wheat acreage as in 
Phase I and has been expanded to include evaluation of yield 
and production estimates as well. Specifically, the objectives 
of the Phase II AA program are: 

a. To establish experimental designs and conduct analyses which 
allow 

• Computation of the variance, bias, and confidence limits 
for LACIE estimates of acreage, yield, and production. 

© Identification of LACIE errors to a level of detail 
sufficient to determine the source and magnitude of 
component error contributions and, if possible, to 
recommend solutxons for correction of the related 
problems . 

a Investigation of problems identified in the Information 
Evaluation (IE) Monthly Report (IMR) . 

b. To define the requirements for gathering ground truth, sat- 
ellite imagery, aircraft photography, and other ancillary 
data necessary to support the accuracy and reliability 
assessment of LACIE estimates of acreage, yield, and pro- 
duction. This task involves monitoring the LACIE acquisi- 
tion process to ensure the timely application of program 
resources in the AA data-gathering process. 

c. To report on the experimental findings (in format reports 
and in informal briefings to project personnel) in a manner 
that will support the preparation of LACIE project evalua- 
tion reports. 
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3 . 2 SCOPE 


The Phase II AA program is designed to assess and evaluate, in a 
quasi-operational mode, the capabilities of LAC IE to develop 
wheat acreage, yield, and production estimates for the United 
States that will meet the 90-90 criterion in the U.S. Great 
Plains. The AA process requires the multiagency collection and 
subsequent evaluation by the LAC IE of the following data: 

a. LACIE imagery and classification data. 

b. Aerial photography (the basis for development of field maps) . 

c. Field maps and land-use annotations developed from USDA 
Agricultural Stabilization and Conservation Service (USDA/ 
ASCS) ground observations. 

d. Inventories of all fields after fall planting for winter 
wheat areas and after spring planting for both spring and 
winter wheat areas . 

e. Periodic observations of ITS's within 3 days of each Landsat 
overpass (every 18 days) . 

These data will be collected 

• From 40 blind sites in the U.S. southern Great Plains for 
early-season and at-harvest evaluations. 

• From 136 blind sites throughout the U.S. Great Plains for at- 
harvest evaluations . 

• From 27 ITS's throughout the U.S. Great Plains for evaluations 
throughout the growing season. 

Listings of ITS's and blind sites are presented in appendix B. 
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The AA Team will monitor the processing of the above data set 
for the United States and compare: 

a. LAC IE estimates of acreage, yield, and production with those 
obtained from ground observations by the USDA/ASCS over the 
above sites. 

b. LACIE estimates of acreage, yield, and production at the Crop 
Reporting District (CRD) , state, regional, and national levels 
with similar USDA/SRS estimates. 

c. LACIE adjustable crop calendar (ACC) esuimates with USDA/SRS 
wheat-growth-stage information from the ITS ' s . 

In addition, AA personnel will assemble and evaluate similar data 
from 10 Canadian ITS 1 s and compare LACIE estimates in Canada with 
USDA Foreign Agricultural Service (USDA/FAS) reports at the 
national level. 
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4. GENERAL TECHNICAL APPROACH 


Three groups of activities are required to implement the LACIE 
Phase II AA and satisfy its objectives: 

a. Data requirements definition and acquisition monitoring 

b. Data analysis and evaluation 

c . Reporting 

A detailed flow diagram of the Phase II AA program and related 
LACIE operational activities is presented in figure 4-1. Spe- 
cific descriptions of the AA tasks associated with each group 
of activities are presented in section 6. 

4.1 DATA REQUIREMENTS DEFINITIONS AND ACQUISITION MONITORING 

The first group of the Phase II AA activities involves the iden- 
tification of data requirements to support the accuracy evalua- 
tions of the various LACIE component products and the monitoring 
of data acquisitions by the LACIE operational organization and 
related NOAA and USDA functions. These activities invo lve the 
identification of LACIE operational data products to be used in 
AA, the definition of methods by which these products can be 
retrieved in a timely manner from LACIE operations for AA analy- 
ses , and identification of requirements for reference and control 
data from NOAA and USDA. 

AA Team members will (1) select blind-site data at random 
(approximately 40 sites for early- and late-season evaluations 
and approximately 136 sites for late-season evaluations) from 
segments which have at least one Landsat acquisition and which 
have been processed by the CAMS and forwarded to the Crop Assess- 
ment Subsystem (CAS) and (2) coordinate action for acquiring 
ground truth from the blind sites and ITS's in the United States 
and for retrieving CAMS classification data for these sites from 
LACIE operations. The locations of the blind sites are withheld 
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from the CAMS AI’s so that these segments will be processed as 
regular operational segments. 

Although AA data acquisition is largely accomplished by LACIE 
operations personnel, extensive coordination will be required by 
AA in order to ensure the timely and accurate selection of blind 
sites and the gathering of adequate evaluation data. 

The specific tasks to implement the above activity are identified 
as tasks 1 through 8 in figure 4-1. Specific descriptions of 
these tasks are provided in section 6.1. 

4.2 DATA ANALYSIS AND EVALUATION 

The second group of activities of Phase II AA concerns the analy- 
sis and evaluation of the basic data collected during the initial 
AA activities. Comparative statistics are developed to support 
evaluations of LACIE acreage, yield, and production estimates. 
This will require the processing of subsets of these data, which 
are appropriate for the specific AA evaluation being conducted. 
Appropriate statistics are calculated to satisfy the experimental 
queries in the evaluations. 

Phase II AA evaluations are designed to examine the LACIE opera- 
tional products (estimates of acreage, yield, and production) 
and related errors over the U.S. Great Plains. This includes 
the win ter-wheat states (the U.S. southern Great Plains; i.e., 
Colorado, Kansas, Nebraska, Oklahoma, and Texas) and the spring- 
wheat states (the U.S. northern Great Plains; i.e., Minnesota, 
Montana, and North and South Dakota) . 

In order to accomplish its objectives of determining the magnitude 
and components of error in LACIE estimates and of ascertaining 
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whether or not the LACIE is satisfying the 90-90 criterion, AA 
will: 

a. Determine the error in LACIE estimates of acreage, yield, and 
production. 

b. Establish an error budget and compute the coefficients of 
variation (CV's) of LACIE production estimates and compare 
them to the budgeted values to determine if the 90-90 crite- 
rion is being met. 

c. Through detailed investigation, examine the potential sources 
of error within the LACIE estimates. This includes evaluat- 
ing acreage estimation error sources within classification, 
sampling, and aggregation activities. Yield and production 
error sources will also be evaluated. If possible, these 
studies should provide recommendations of ways and methods 

to reduce the error. 

d. Investigate the various error types and the magnitude of 
their effects on LACIE performance through development and 
usage of a LACIE Performance Predictor (LPP) model in evalu- 
ative simulations of the LACIE operational system. 

The specific methods for implementing these tasks are identified 
as tasks 10 through 35 in figure 4-1. Specific descriptions of 
these tasks are provided in section 6.2. 

4.3 REPORTING 

Reporting for Phase II AA will consist of the following three 
reporting methods: 

1. Special AA management briefings and presentations 

2 . AA monthly quick-look reports ' 

3. Phase II AA Report 
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AA will provide LACIE management with special briefings and 
presentations on the status of AA data acquisitions and special 
problems during the 1975-76 winter- and spring-wheat growing 
seasons. These briefings provide timely 'responses to management 
requests for visibility into LACIE processes throughout the 
growing season and support to LACIE management in providing 
information on current LACIE status and problems to ..upper-level 
NASA, USDA, and NOAA management. 

AA will also provide monthly quick- look reports to LACIE manage- 
ment consisting of comments on current CAS Monthly Reports (CMR’s) 
throughout LACIE Phase II. The comments will be the result of AA 
cursory reviews of these reports. The monthly quick-look reports 
will provide technical comments on the data presented and report 
on special problems and the results of any investigations of these 
problem areas conducted by AA. Some of these problems may be the 
subject matter of planned Phase II AA investigations of LACIE 
error sources. If the nature of the problems identified in the 
CMR and IMR reviews warrants special studies, they will be ini- 
tiated as part of the Phase II AA investigations. 

In addition to the monthly quick-look reports, AA will prepare 
a Phase II AA report developed through a series of interim 
reports that are prepared throughout Phase II. The first interim 
report consists of evaluations of early-season winter-wheat esti- 
mates in the U.S. southern Great Plains. The second interim 
report examines early- and late-season winter-wheat estimates 
in the same area. The third interim report provides AA evalua- 
tions of at-harvest estimates of U.S. winter and spring wheat 
over the U.S. Great Plains. A fourth interim report will be 
developed to serve as a draft of the final Phase II report. It 
will be reviewed and revised to become the final Phase II AA 
report. 
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This iterative building-block process of preparing the Phase II 
AA report has been developed to allow a thorough review and 
critique of each report, with the technical comments being used 
to upgrade and improve the succeeding drafts of the report. It 
also provides for the analysis and addition of technical data 
that become available during the interim period between report 
drafts . 

A detailed description of the basic reporting formats and the 
suggested content of these reports are provided in the detailed 
task descriptions presented in section 6.3 of this plan. 
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Schematic diagram of LACIE Phase II Accuracy Assessment. 
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5. SCHEDULE AND RESOURCE REQUIREMENTS 


The schedule and resource requirements for implementing LACIE 
Phase II AA are presented in the following sections. 

5.1 SCHEDULE 

The Phase II AA schedule is presented in figure 5-1. 

5.2 RESOURCE REQUIREMENTS 

The resource requirements for LACIE Phase II AA are summarized 
in table 5-1/ which summarizes the manpower and computer require- 
ments associated with specific AA tasks or task groups. 



TABLE 5-1.- RESOURCE REQUIREMENTS FOR LAC IE PHASE II ACCURACY ASSESSMENT 




Manpower requirements 
(man-year equivalents) 

Computer requirements 

Task 

number 

Task or task 
group title 

Management 

Agronomy/ 

climatology 

Computer 

programming 

Clerical 

assistants 

CAMS 

analysis 

Statistical 

analysis 

Accuracy 

team 

Cartography 

UNIVAC 1108 
Exec II (hrs) 

UNIVAC 1110, 
Exec VIII (hrs) 

w 

■d 

rd 

co 

Pm 

H 

0$ 

w 

IMAGE 100 (hrs) 

1-8 

AA planning and 

data-gathering 

tasks 

2.9 



1.0 

1.0 


1.2 

1 . 5 





9-35' 

i 

AA data analysis 
and evaluation 
tasks 

1.7 

1.5 

2.0 

.2 

7.0 

4.5 

.6 


75 

15 

500 

(normal 

operations 

344) 

40 

36-3 8 

AA reporting 

.4 

.3 


1.0 


1.5 

.2 







Total Phase II 
requirements 

5.0 

1.8 

2.0 

2.2 

8.0 

6.0 

2.0 

1.5 

75 

15 

500 

(normal 

operations 

344) 

40 


Earth Resources Interactive Processing System. 

^Interactive Multispectral Image Analysis System, model 100. 
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6. TASK DESCRIPTIONS 


Detailed descriptions of the tasks that comprise AA for Phase III 
are included in the following subsections. 

6.1 PLANNING AND DATA ACQUISITION 

One of the three major elements of Phase III AA is the planning 
and coordination of analytical and data acquisition activities 
of the program. This involves (1) the determination of the basic 
data requirements that will satisfy the analytical and evalua- 
tional requirements of the program, (2) the coordination of data 
acquisition resources so that the timely acquisition of data is 
accomplished, and (3) the monitoring of data quality to assure 
that the data acquired are of satisfactory technical quality for 
assessing the accuracy of the LACIE estimates of acreage, yield, 
and production. 

The AA program depends on the LACIE functional elements (CAMS , 

CAS, YES, and DAPTS) to provide a majority of the data necessary 
for evaluations. Specific task descriptions are not included in 
the AA plan for this LACIE operations data development and pro- 
visioning effort. Only the AA data acquisition tasks where direct 
involvement of the AA Team or support personnel is required are 
identified and described in the following paragraphs . 

6.1.1 PLAN DEVELOPMENT 

The initial activities of Phase II AA are directed to the develop- 
ment of an AA plan. This involves definition of the program 
scope, data/resource requirements, scheduling, and task descrip- 
tions of the activities planned for Phase II. The planned 
activities of the Phase II AA program are documented in this 
Phase II AA plan. 
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6.1.2 DATA REQUIREMENTS 

Required inputs from the LAC IE functional elements have been 
identified through the planning for AA investigations. The 
basic inputs are defined in the following subsections. As 
special problems are identified, these requirements may be 
expanded to satisfy newly defined AA needs. 

6. 1.2.1 Classification and Mensuration Subsystem 

The following data and information are required from the CAMS 
records for all 1975-76 U.S. blind sites and ITS's (see 
appendix A) . 

a. A copy of the CAMS evaluation form for every acquisition and 
every run 

b. A standard classification summary and statistical report for 
every run 

c. The printed output of the classification-on-microfilm (COM) 
tape for each batch run 

d. Printer outputs for each run when the CAMS-CAS interface 
tape becomes operable 

e. A classification map 

f. Training- and test-field coordinates for every run 

g. A record of the AI interpretation of the biostage 

All of these items shall be collected by segment number and LAC IE 
crop year and stored so that all items from each run can be 
easily collected by AA for use in performing the analyses 
defined in section 6.2 of this plan. 

In addition to the above, full-frame Landsat multispectral 
scanner (MSS) color infrared imagery is required from several 
counties in each U.S. Great Plains state for photointerpretation 
of every segment in each county selected. These counties will 
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be defined in the LAC IE sampling plan document and utilized by 
the Sampling Team to provide AA with evaluations of within-county 
variance. 


6. 1.2. 2 Crop Assessment Subsystem 

The following data and information inputs are required from CAS 
during Phase II AA: 

a. CMR 1 s , which include LACIE Phase II estimates of acreage, 
yield, and production as they are developed throughout the 
growing season. 

b. The standard statistics for LACIE Phase II estimates of wheat 
acreage, yield, and production, which include the standard 
deviation (a) , the CV, the 90 -percent confidence limits, and 
the probability of a 10-percent relative error. 

6. 1.2.3 Yield Estimation Subsystem 

The following data and information inputs are required from the 
YES during the Phase II AA: 

a. ACC data over the 27 ITS’s in the U.S. Great Plains are 
required periodically (after completion of each growth stage) 
throughout the winter- and spring-wheat growing seasons , and 
USDA/ASCS estimates are required for the same periods in the 
following CRD ' s : 

« Texas (IN, IS, 2N, 2S, 5N, 5S , 8N, and 8S) 

• Kansas (all CRD’s) 

• North Dakota (all CRD’s) 

• Montana (1, 2, and 3) 

b. Meteorological data, yield estimates, and standard statistics 
are required for each stratum for specific months as indicated 
in the schedule which is table 6-1. 
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6. 1.2. 4 Data Acquisition, Preprocessing, and Transmission 
Subsystem 

The following data and information inputs are required from the 

DAPTS during Phase II AA. 

a. The following ground-observation data for all U.S. blind 

sites: 

• A.ircraft photography (the basis for development of field 
overlays that are used in documenting inventories and 
interpreting signatures; see section 6.1.4 for specific 
requirements) . 

e Completed spring early-season inventories of the blind 
sites conducted by USDA/ASCS personnel using the instruc- 
tions and data recording forms presented in appendix D . 

• Completed at-harvest inventories of the ITS ' s conducted 
by USDA/ASCS personnel utilizing the data forms presented 
in appendix E. 

b. The following ground-observation data (to be collected as a 

part of LACIE operations) for all U.S. ITS's: 

• Aircraft photography (the basis for development of field 
overlays that are used in documenting inventories) . 

• Completed fall and spring early-season winter-wheat 
inventories of the ITS's conducted by USDA/ASCS personnel 
utilizing the data forms presented in appendix E. 

• Completed 18-day periodic observations of the ITS's 
throughout the wheat growing season until harvest, to be 
taken within 3 days of Landsat overflights. (The USDA/ 
ASCS will make and record these observations on the forms 
presented in appendix E.) 
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® Completed postharvest yield inventory of the ITS's con- 
ducted by USDA/ASCS personnel utilizing the data forms 
presented in appendix E . 

c. Copies of mean historical crop calendars based on the last 

15 years, if available, for each acreage stratum for the U.S. 
Great Plains states and Canada. 

d. Agriculture reports: 

• USDA/SRS reports containing current information on wheat 
acreage, yield, and production for the United States, 
including aggregation data at the state, regional, and 
national levels and yield data at the CRD, state, regional, 
and national levels . These data are needed on the day the 
reports are released in order to support the quick- look 
report. 

a USDA/FAS reports containing current year information on 
wheat acreage, yield, and production. 

e. Historical agricultural statistics 

• USDA/SRS data on wheat acreage, yield, and production in 
the United States for 1970-75. 

• USDA Agricultural Census data for 1969 and 1974. 

f. Other required data sets, as specified by AA to satisfy 
special investigations that may be requested by LAC IE 
project management. AA will specify any such requirements 
to DAPTS as soon as possible after data requirements are 
identified. 

6.1.3 BLIND-SITE SELECTION 

A random sample of blind sites will be selected by the AA Team 

.from LACIE U.S. segments which have at least one Lands at acqui- 
sition processed by CAMS and forwarded to CAS. 
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6. 1.3.1 Early Season 

The early-season blind sites are selected after release of the 
CMR for February 1976. Appendix B (table B-3) indicates the 
states in the U.S. southern Great Plains from which the early- 
season blind-site selection is made and the expected number of 
blind-site samples from each of these states. 

6. 1.3. 2 At Harvest 

Blind sites for the at-harvest evaluation will be randomly 
selected for each state from sites having at least one Landsat 
acquisition processed by CAMS and forwarded to CAS . Appendix B 
(table B-4) contains a description of the expected number of at- 
harvest blind sites for each state and additional information 
about the at-harvest sites. The 40 sites used for the early- 
season evaluation will also be used for the at-harvest 
evaluation. 

6.1.4 PREPARATION OF BLIND-SITE FIELD OVERLAYS 

In order to obtain valid, useful, and documented ground- truth 
information for use in the Phase II AA evaluations, field over- 
lays must be prepared from aircraft photographs of the blind 
sites. These field overlays are then used to record the land- 
use information that supports the acreage determination aspect 
of the ground-observation inventories conducted throughout LACIE 
Phase II. The following items are required to prepare the blind- 
site field overlays. 

6. 1.4.1 Aircraft Maps 

After selection of the blind sites, Landsat imagery is used to 
determine the true position of each site. AI’s determine these 
positions using production film converter (PFC) products, record 
the latitude and longitude to the nearest 0.1 minute, and plot 
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the position of the segment on a 1:24 000- or 1:12 500-scale map. 
These maps are then used by aircrews in acquiring the aerial 
imagery. 

6 . 1 . 4 . 2- Aircraft Photography 

Photography will be collected using RC-8 infrared film from 
aircraft at an altitude of 6000 to 7200 meters {20 000 to 
24 000 feet) . Two flight lines will be flown for each site with 
a 20-percent sidelap. Four frames will be collected for each 
flight line with a 30-percent forward overlap. All imagery must 
be collected no later than 4 weeks prior to ground-truth collec- 
tion. Predesignated flight lines will be established by the 
LAC IE for each blind site. 

After aerial imagery is acquired for the blind sites, each frame 
will be checked to verify that the site was covered and that the 
imagery is of sufficient quality to be used by the USDA/ASCS 
personnel in collecting ground truth. 

6. 1.4. 3 Field Overlays and Field Segment Kits 

If the imagery is of satisfactory quality to be used by USDA/ASCS 
personnel, transparent overlays are prepared outlining and iden- 
tifying fields. The overlays are then placed in field segment 
kits that are forwarded to USDA/ASCS personnel for use in 
acquiring ground truth. These kits include: 

• A color infrared 2X print of the segment with field overlay 

• A topographical map of scale 1:250 000 showing the sample 
segment location and boundaries 

• Crop identification key (standard annotation to be used in 
documenting land use) 

9 Survey manual — a brief definition of field procedures 

developed at JSC providing guidelines to USDA/ASCS personnel 
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as to the basic project requirements for recording ground 
observations of the LAC IE blind sites (appendix D) 

6.1.5 BLIND-SITE FIELD DATA ACQUISITION 

USDA/ASCS personnel provide ground observations, annotations of 
field overlays, and complete inventory data. Fields" are annotated 
according to the standard crop symbols identified in the crop 
keys provided in the JSC instructions to USDA/ASCS for making 
LACIE segment inventories (appendix D) . The inventory package 
is completed by USDA/ASCS personnel and forwarded to JSC to be 
logged and tracked by DAPTS. 

6.1.6 DETERMINATION OF THE WHEAT/SMALL- GRAIN PROPORTIONS FOR 
BLIND SITES 

The LACIE cartographic technician plots the Landsat scene 
product 1 boundary on a 2X photograph. Using the area mode 
feature of the H. Dell Foster digitizer, the technician measures 
the segment area in thousandths of a square inch on the photog- 
raphy. Next, the area of each wheat and small-grain field is 
measured and the sum of the individual wheat/ small-grain fields 
is divided by the total area of the segment. Likewise, all 
abandoned wheat/small-grain fields are planimetered, and the sum 
is divided by the total area. The result will be the percentage 
of both wheat and small grains. 

6.1.7 FIELD DATA ACQUISITION IN INTENSIVE TEST SITES 

The field data acquisition from 27 U.S. and 10 Canadian ITS ' s is 
an integral part of LACIE operations. These sites are located 
prior to Phase II operations , and their identities and locations 
are available to all LACIE personnel (see appendix B) . Field 
data acquired from these sites by USDA/ASCS personnel include 
the following: 

a. Aerial photography (once yearly) 
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b. Field maps annotated by USDA/ASCS personnel 

c. Inventories of all fields (appendix E provides examples of 
data reporting forms) : 

• After fall planting and before harvest for winter -wheat 
areas 

• After spring planting and before harvest for both- spring- 
and winter-wheat areas 

d. Periodic 18-dav observations of a subsample (approximately 
50 fields) of each ITS coincident with each Landsat overpass 
(Appendix E gives an example of a form for recording these 
18-day periodic observations.) 

These data are forwarded to JSC to be processed and logged by 
DAPTS. DAPTS then furnishes this information to AA for 
evaluation. 

6.1.8 DETERMINATION OF WHEAT/SMALL-GRAIN PROPORTIONS FOR 
INTENSIVE TEST SITES 

As a part of LACIE operations , the wall-to-wall fall and spring 
inventories are accomplished, and the crop proportions are cal- 
culated for the ITS's in the manner described in section 6.1.6 
for the blind sites. 

6.2 ANALYSIS AND EVALUATION 

The LACIE Phase II AA function designs and conducts experimental 
evaluations of LACIE estimates of acreage, yield, and production. 
The purposes of these investigations are (1) to determine error 
in these estimates, (2) to ascertain whether the error is within 
acceptable limits for meeting the LACIE 90-90 criterion, and 
(3) to conduct investigations designed to identify possible 
sources of the error or, in other words, to identify or point 
to potential causes of the variation- in LACIE estimates from the 
actual or reference (USDA/SRS) data. 
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The AA analysis and evaluation function, using data systematically 
gathered over the yardstick region (U.S. Great Plains), is 
directed toward drawing inferences as to the performance of 
LAC IE technology if operated for a span of several years. 

The following analysis and evaluation tasks and activities are 
those presently planned to accomplish the LACIE Phase II AA pur- 
poses noted above. Detailed descriptions of these tasks are 
presented in the following subsections. 

6.2.1 DATA COLLECTION AND DEVELOPMENT OP COMPARATIVE STATISTICS 

This task is an integral part of the analytical activities iden- 
tified in the following subsections. However, it has been speci- 
fied as a separate task because of the time demands placed on AA 
analytical personnel to acquire the data from various locations 
within LACIE operations and the LACIE Physical Data Library 
(LPDL) . Upon acquisition of these data, the AA Team processes 
and develops statistics to satisfy the requirements of the AA 
experimental evaluations . 


6.2.2 DETERMINATION OF ERROR IN LACIE ESTIMATES 

A prime concern of the AA program is to monitor and status the 
development of LACIE estimates throughout the growing season. 

This task requires review and evaluation of LACIE acreage, yield, 
and production estimates. The evaluations are made largely from 
comparisons with: (1) similar estimates made over the same state 

or region during similar times in the growing season by the USDA/ 
SRS and (2) ground truth obtained from ground observations of 
special sites (blind sites and ITS’s). 

The error in LACIE estimates is described by the calculated rela- 
tive difference ( ) obtained in these comparisons. 
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Also , states or regions where significant differences are noted 
will be further evaluated to determine probable causes of these 
errors . 

These comparisons are designed to detect, early in the growing 
season, any divergences between the estimates and reference 
standards and thus to identify for further investigations poten- 
tial LACIE problem areas that might be associated with the 
divergences . 

The results of these reviews and evaluations will be reported in 
the Phase II AA monthly quick- look reports, which are described 
in section 6.3.2. 

6.2.3 DETERMINATION IF LACIE PRODUCTION ESTIMATES MEET THE 
90-90 CRITERION 

The procedures for determining if the LACIE production estimates 
for the U.S,. Great Plains are meeting the 90-9 0 criterion are set 
out in appendix C. These procedures show that, if the production 
estimate is unbiased and if the CV for production, CV(W), at the 
nine-state level is less than 6 percent, the 90-90 criterion will 
be met. In general, it is first assumed that the production 
estimate is unbiased. An alternate approach is to assume that 
the USDA/SRS production estimates are correct and to define the 
bias to be the difference between the LACIE production estimate 
and the USDA/SRS estimate. In this case, the tolerable CV{W) 
would be lower than 6 percent. This approach will be investigated 
during Phase II but probably will not be used operationally to 
determine if the LACIE is meeting the 90-90 criterion. 

The method used to estimate the CV for production at the nine- 
state level is presented in appendix C. 
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6.2.4 INVESTIGATION OF LAC IE ERROR SOURCES 


After determining the error in LACIE estimates and whether or not 
the estimates are meeting LACIE goals, AA will conduct further 
investigations to isolate and identify the error sources or poten- 
tial causes of errors in LACIE estimates. LACIE Phase I AA 
experience has identified a number of potential error sources 
associated with the acreage, yield, and production estimates. 

These will be investigated further in Phase II , along with other 
likely potential error sources that are identified throughout the 
analyses conducted during Phase II. The specific tasks involved 
in these error source investigations are described in the follow- 
ing subsections. 

6. 2.4.1 Production Error Source Evaluations 

The error sources within LACIE production estimates are examined 
by investigating the following specific tasks. 

6. 2. 4. 1.1 Comparisons of LACIE and USDA/SRS Estimates 

This task involves the analysis of LACIE production estimates 
through comparisons with USDA/SRS estimates and evaluations at 
the county, CRD, state, regional, and national levels. Statis- 
tical analyses are performed during the growing season using the 
standard statistics produced by CAS. Graphical analyses at the 
state, regional, and national levels are conducted throughout 
the season to determine if consistent and predictable trends in 
the development of the estimates can be established. Year-to-year 
comparisons at the state level are also made in areas where con- 
sistent growing conditions have existed. 
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6.2. 4.1.2 Other LAC1E Production Estimates 


In addition to the above assessments of production error sources, 
AA examines general reasons for the error in LACIE production 
estimates. These special investigations, as required, will be 
fully documented along with the specific evaluations of error 
sources wirhin the 'acreage and yield estimates , which are 
described separately in section 6. 2. 4. 2. 

6. 2. 4. 2- Acreage Error Source Evaluations 

The error sources within LACIE acreage estimates are evaluated 
with respect to the specific classification and sampling error 
evaluation tasks described in the following subsections. 

6. 2. 4. 2.1 Classification Accuracy 

Error sources associated with the LACIE process of classifying 
the imagery and determining the proportion of the area that is 
wheat will be examined through a series of investigations of: 

a. Temporal effects (tasks 14 through 16, figure 4-1) 

- b. AI procedural effects (tasks 17 through 20, figure 4-1) 

c. Crop calendar effects (tasks 21 and 22, figure 4-1) 

The specific investigation tasks to determine classification 
error sources are described as follows. 

6. 2. 4. 2. 1.1 EARLY-SEASON PERFORMANCE EFFECTS . This task will 
be conducted to ascertain error sources within CAMS estimates of 
wheat proportions associated with the early phases of the winter- 
and spring-wheat growing seasons. 

The basic data set for these investigations is the CAMS opera- 
tional classification results (as defined in section 6.1. 2.1) and 
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the ground-truth wheat proportion estimates from blind sites and 
ITS’s. The early-season winter-wheat evaluation sites are located 
in the U.S. southern Great Plains whereas the early-season spring- 
wheat evaluation sites are located in the U.S. northern Great Plains. 

The locations of and descriptive data on the LAC IE ITS's and blind 
sites to be used in these investigations are presented in appen- 
dix B. (ITS’s in the States of Washington , Idaho, and Indiana 
are listed also but will not be utilized in these early-season 
evaluations) . To ensure the processing of the blind-site seg- 
ments as regular operational- sites, specific descriptive data 
such as blind- site locations are maintained as secure data until 
after completion of the processing of all Phase II segments. 

Evaluations are made by comparing LACIE wheat proportion estimates 
with those obtained from the ground-truth observations. Relative 
differences and the standard errors of the relative differences 
are calculated from these comparisons. The mean errors are then 
determined for each state and for the five-state early-winter- 
wheat and four-state early- spring-wheat areas. From these data 
the AA Team estimates the bias, variance, and confidence limits 
for LACIE estimates. 

In addition to the above blind-site and ITS investigations, 
temporal plots of wheat estimates for the U.S. Great Plains 
states are developed as part of task 6.2.2 for the early-season 
evaluations and are continued throughout the growing season. 

These plots, which are made by state and aggregated for the U.S. 

Great Plains, compare LACIE and USDA/SRS acreage estimates. An 
examination of these data will disclose trends in the development 
of the estimates and error source inf ormation . 

6. 2. 4. 2. 1.2 LATE-SEASON PERFORMANCE EFFECTS . Basically, this 
task is the same as the early-season performance effects task. 
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described in section 6. 2. 4. 2. 1.1, except that a data set including 
acquisitions during the late winter-wheat and late spring-wheat 
seasons is used. In addition, the temporal plots are expanded to 
include LACIE and USDA/SRS acreage estimates for the' total growing 
season. These data provide visibility into the trends, if any, 
that exist in the development of acreage estimates during the 
season. 


6. 2. 4. 2. 1.3 NUMBER OF ACQUISITIONS . This task involves evalua- 
tions to determine the effects upon classification of the number 
of acquisitions obtained over a segment throughout the growing 
season. 

The investigations will utilize CAMS classification and ground- 
truth data from acquisitions obtained over the 176 U.S. blind 
sites during the 1975-76 growing season. 

The purpose of these evaluations is to see if the observed pro- 

A 

portion errors (P - ? CT ) are correlated as a function of biowindow 
with the number of acquisitions or combinations of acquisitions 
over a segment during the growing season. 

Proportion errors will be correlated with those segments having: 

a. One acquisition (biowindow 1, 2, 3, or 4) 

b. Two acquisitions (biowindows 1 and 2, 1 and 3, 1 and 4, or 
2 and 3) 

c. Three acquisitions (biowindows 1, 2, and 3; 1, 2, and 4; 

1 , 3 , and 4 ; or 2 , 3 , and 4 ) 

d. Four acquisitions 

Patterns of acquisitions and the relationship of error magnitude 
to acquisition pattern will be examined. 
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6. 2. 4. 2. 1.4 AI FIELD LABELING ACCURACY . The purpose of this 
task is to investigate the degree of relationship that exists 
between observed proportion error and error in the labeling of 
fields by the AI. 

CAMS classification data and ground-truth observations over the 
176 U.S. blind sites and 27 U.S. ITS's are the basic data for 
these correlations. 


This task is designed to establish the relationship of field 
labeling error to classification error and, if possible, to 
establish a threshold in field labeling error above which an 
unacceptable CAMS proportion error will probably occur. 

6. 2. 4. 2. 1.5 AI DIFFERENCES . This task examines the AI classi- 
fication error and determines if any pattern of differences 
can be noted in AI performance. 

The basic data set for these investigations is CAMS classifica- 
tion data over the U.S. Great Plains blind sites throughout the 
1975-76 growing season compared with USDA/ASCS ground-observed 
proportions over the same 176 blind sites. 


AI performance will be evaluated by determining the relative 

f A 

P - P. 


difference 


GT 


for each AI over each blind site. These 


are averaged over all the U.S. Great Plains blind sites to obtain 

fx - 
P - P. 


a mean relative difference 


GT 


along with a standard 


deviation (a) for this distribution. This will provide a pre- 
liminary assessment of AI error in assessing wheat proportions 
as well as a measure of variance noted about the mean relative 
difference value. Further investigations will be made, as 
required by an evaluation of these results. 
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6.2. 4. 2. 1.6 NUMBER OF TRAINING FIELDS . This task assesses the 
relationship between the number of training fields selected by 
the AI and the CAMS proportion errors. In addition , the rela- 
tionship between the number of pixels selected by the AI for 
training and proportion errors is examined in this task. 

The basic data set for this task is the same as set out in the 
AI field labeling accuracy task (section 6. 2. 4. 2.1. 4) , the CAMS 
classification data and ground-truth observations over the 176 
U.S. Great Plains blind sites and 27 U.S. ITS's. 

The evaluations (1) provide information as to how the number of 
fields selected by the AI for training relates to proportion 
error and (2) examine the relationship of the number of pixels 
or total training area to proportion error. It may be possible 
from these investigations to establish thresholds for numbers of 
fields and/or pixels required for adequate proportion estimation 
accuracy. 

6. 2. 4. 2. 1.7 TOTAL SEGMENT WHEAT PROPORTION EFFECTS . This task 
evaluates the relationship between the probability of correct 
classification (PCC) and the proportion of wheat harvested (P^.) . 


The basic data set is the CAMS classification results and the 
ground- truth observations from test fields selected from the 
27 ITS's throughout the United States. 

From these data, the PCC of wheatfields is calculated along with 

the P obtained for the same fields. A regression of the PCC on 

P T7 for these ITS's is made on a per-segment and a per-field basis, 
w 

These evaluations should provide some further understanding of 
the underestimation in drought areas and establish if biases in 
LACIE estimation could be related to the proportion of wheat 
that was harvested. 
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6. 2. 4. 2. 1.8 CAMS EVALUATION CODES . This task provides for a 
general investigation of the differences in the CAMS -evaluated 
classifications of "marginal" and "satisfactory." 

The basic data set is the CAMS classification data and the 
ground-truth observations over the 176 U.S. blind sites and, if 
warranted, over the 27 U.S. ITS's. 

Proportion errors for those CAMS classifications evaluated as 
"marginal" and "satisfactory" will be determined over represen- 
tative blind-site acquisitions. Differences between these cate- 
gories of CAMS evaluation codes will be calculated and evaluated. 

6.2. 4. 2. 1.9 CROP CALENDAR ERROR DETERMINATION . Major reference 
data utilized by Al's in their classification procedures are the 
nominal {mean historical) crop calendar and the ACC. Since the 
LAC IE ACC developed by CCEA provides the latest reference infor- 
mation on the stage of development of wheat in an area being 
classified and estimated, it is necessary to determine the accu- 
racy of this reference information. This task is designed to 
determine the accuracy of the ACC estimates of the wheat growth 
stage throughout the growing season. 

A 

The basic data set for these evaluations is the growth-stage 
data acquired by the USDA/ASCS personnel over the 27 U.S. ITS's. 
These growth-stage data are acquired utilizing the ground-truth 
periodic observation form presented in figure E-2 (appendix E) . 

In addition, growth-stage information is acquired by the USDA/ASCS 
for each of the following CRD's: 

a. Texas (IN, IS, 2N, 2S , 5N, 5S, 8N, and 8S) 

b. Kansas (all CRD's) 

c. North Dakota (all CRD's) 

d. Montana (1, 2, and 3) 
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The CRD growth- stage data are acquired by USDA/ASCS personnel 
using the growth- stage reporting forms presented in appendix F. 

The USDA/ASCS delivers these data to the DAPTS at JSC 30 days 
after completion of each applicable growth stage in each CRD 
(stratum) or equivalent. 

From these data, comparisons are made between ACC outputs (at 
the CRD level) and ground observations of wheat growth stages 
averaged to a CRD level throughout LACIE Phase II. Differences 
between ACC output and ground observations are recorded as days 
"ahead of" or "behind" the ground observations. Examinations of 
the variance of the ACC output over the U.S. Great Plains will 
be made. The relationship of this information to known episodic 
events of the current year, such as drought, is investigated and 
reported throughout Phase II AA, along with the assessment of the 
accuracy of the ACC. 

6.2.4.2.1.10 CROP CALENDAR EFFECTS ON CLASSIFICATION AND 
LABELING ERRORS . A concern of AA has been to determine if the 
AI ' s use of the ACC results in more accurate wheat proportion 
estimates than those obtained using the nominal (mean historical) 
crop calendar. Since the Al's use both the nominal crop calendar 
and the ACC, it is difficult to assess the weight that is given 
to each. 

The basic data set for evaluating the ACC versus the nominal crop 
calendar results will be the growth-stage data collected from 
the 27 U.S. Great Plains ITS 1 s . In this task, scatter plots are 
prepared in which the abscissa is the CAMS proportion error and 
the ordinate. is the amount, measured in days, by which the ACC 
is "better than" or "worse than" the nominal crop calendar. 
"Better than" means that the ACC is closer to the ground-truth 
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growth-stage data for the ITS's for the 1975-76 growing season 
(which are assumed to be error free) . The scatter plots will be 
studied to disclose any correlation between CAMS proportion error 
and the degree to which the ACC varies from the nominal crop 
calendar, using ground rruth as reference data. 

6.2.4.2.1.11 EVALUATION OF CANADIAN INTENSIVE TEST SITE 
CLASSIFICATIONS . The previous classification tasks have addressed 
the major thrust of Phase II AA; that is, to evaluate LACIE esti- 
mates over the U.S. Great Plains. The AA evaluations in this task 
are extended to investigate the accuracy of LACIE estimates using 
the ACC over the Canadian ITS’s (appendix B, table B-2) . 

The data set for these investigations will be LACIE classification 
data and ground-truth data obtained in fall inventories and in 
periodic 18-day observations over the 10 Canadian ITS's. Crop 
growth-stage data over these ITS's are used, also, in these 
investigations . 

The ACC error will be determined in a manner similar to that used 
for the U.S. Great Plains ITS's (see section 6. 2. 4. 2.1. 9) . In 
addition, the LACIE wheat/small-grain proportion estimate (P) 
over each ITS is compared with the ground-truth proportion (P^) 
over the same site. The relative difference is then calculated 
and examined to assess the accuracy of the LACIE acreage estimate. 

In addition, the PCC is examined for the 10 ITS's by selecting 
30 fields at random in each ITS and determining the number of 
pixels classified correctly and those classified incorrectly. 

From these data the PCC for the Canadian ITS's is estimated. 

6.2.4.2.1.12 OTHER CLASSIFICATION EVALUATIONS . As required 
throughout Phase II AA evaluations, special problem areas not 
previously designated as potential classification error sources 
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will probably be identified. As is directed by LAC IE project 
management and the AA Team, special classification error source 
investigations will be conducted (provided resources are available, 
or as such investigations are given special priorities to be per- 
formed in lieu of some previously described classification error 
source evaluation tasks) . 

6. 2. 4. 2. 2 Sampling Accuracy 

The demonstration of the LAC IE project as an economically 
feasible system for making crop inventories dictates that only 
a portion of the wheat growing area be classified. As a result, 
a sampling subset of segments must be utilized and the results 
projected to the total area. The sample segments selected for 
classification must therefore be representative of the entire 
area being classified. Since potential errors can exist in the 
sampling process, it constitutes a major area of concern for the 
AA program. The following Phase II investigations are designed 
to examine the validity of the Phase II sampling method. 

6. 2. 4. 2. 2.1 COMPARISON OF U.S. BLIND-SITE DATA WITH USDA/SRS 
COUNTY ESTIMATES . Since the U.S. Great Plains blind sites con- 
stitute approximately one-third of the 411 LAC IE segments in the 
U.S. Great Plains and since they are randomly selected, they can 
be considered somewhat representative of this region. Evaluative 
information can be gained through a task designed to compare the 
1976 sample-segment wheat proportions with USDA/SRS county wheat 
proportion estimates from the 1969 Agricultural Census. 

The following sampling evaluation tasks will be conducted over 
the U.S. Great Plains by state, by five-state winter wheat, by 
four-state spring and mixed wheat, and by nine-state wheat 
regions. This will involve estimating sampling error by regress- 
ing blind-site segment wheat proportions ( p GT ) on the USDA/SRS 
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1969 historical county wheat proportions and using the 

residual mean square as an estimate of the sampling error (a^) . 

s 


In addition, data are available from the USDA/SRS June enumera- 
tive surveys. These survey data are used for the 1973-74 and 
1974-75 crop years for selected counties, including blind sites, 
in order to determine better estimates of the sampling error 
(including the within-county and between-county error components) . 
These tasks are completed by the Sampling Team. 


6. 2. 4. 2. 2. 2 WITHIN-COUNTY VARIANCE . Formulas exist from which 
sampling variance can be calculated. However, to use these 
formulas, values for within-county variance in the United States 
and other countries are needed. Since all other methods of 
evaluation of sampling variance must be based either upon past 
experience or upon unverifiable assumptions , values of these 
within- stratum or substratum variances must be estimated. 

The Sampling Team estimates within-county variance through the 
usage of full-frame Landsat data. These data are (1) photointer' 
preted to identify agricultural versus nonagricultural areas in 
all 9- by 11-kilometer (5- by 6-nautical-mile) rectangles that 
make up each U.S. Great Plains county and (2) converted to wheat 
percentages using an algorithm developed and tested by the 
Sampling Team. 


6. 2. 4. 2. 2. 3 OTHER SAMPLING EVALUATIONS . As required throughout 
Phase II AA evaluations, special problem areas not previously 
designated as potential sampling error sources may be identified. 
As directed by LACIE project management and the AA Team, special 
sampling error source investigations will be conducted (provided 
resources are available, or as such investigations are given 
special priorities to be performed in lieu of some previously 
described sampling error source evaluation tasks) . 
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6 . 2 . 4 . 2 . 3 Aggregation Accuracy 

The Phase II AA acreage aggregation evaluations consist primarily 
of studies to check the validity of the aggregation software. 

These include the following. 

6. 2. 4. 2. 3.1 USING USDA/SRS COUNTY ESTIMATES FOR LAC IE ACQUISITION 
SEGMENTS . It has been shown that if USDA/SRS county wheat pro- 
portion estimates are substituted for all of the LACIE-al located 
segment estimates and aggregation is made, almost no error occurs 
in the aggregated results when compared with regular USDA/SRS 
estimates. However, as indicated during Phase I, data acquisition 
problems, such as cloud and snow cover and hardware failure, 
resulted in the loss of some of the allocated segments. The 
following procedure will be used to determine if error is occur- 
ring in the wheat acreage estimates at the CRD, regional, and 
national levels because of these missing segments. 

a. Obtain USDA/SRS county estimates for those counties contain- 
ing segments that have been used in the aggregation. 

b. Aggregate to the zonal, regional, and national levels using 
the USDA/SRS county estimates in place of the CAMS estimates. 

c. Compare the aggregated results to the USDA/SRS estimates at 
the zonal, regional, and national levels to determine sig- 
nificant differences. 

d. If differences occur, determine the causes and, if possible, 
make recommendations for corrections in the process . Particu- 
lar attention should be given to the use of the ratio estimate 
for the unsampled group 3 areas. 

e. Compare the aggregated USDA/SRS county results to the aggre- 
gated CAMS results in light of the analyses performed in sub- 
paragraphs c and d. 

f. Issue a report on the effects of aggregation on the 1976 crop 
year data sets. 
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6. 2. 4. 2. 3. 2 USING USDA/SRS COUNTY ESTIMATES FOR LAC IE -ALLOCATED 
SEGMENTS . As noted in subsection 6. 2. 4. 2. 3.1, if USDA/SRS county 
wheat proportion estimates are substituted for all of the LACIE- 
allocated segment values, almost no error occurs between this 
aggregation versus the regular USDA/SRS estimates. When this 
evaluation was made in Phase I, no discrepancies were found. 

This task will be repeated in Phase II to ensure the accuracy 

of the LACIE aggregation software. 

6. 2.4.2. 3.3 OTHER AGGREGATION EVALUATIONS . As required through- 
out Phase II AA evaluations, special aggregation problems not 
previously designated as potential aggregation error sources may 
be identified. As directed by LACIE project management, special 
aggregation accuracy problems will be investigated (provided 
resources are available, or as such investigations are given 
special priorities to be performed in lieu of some previously 
described aggregation error source evaluation tasks) . 

6. 2. 4. 3 Yield Error Source Evaluations 

The two factors in LACIE production estimation are acreage and 
yield. Acreage estimation error source investigations for 
Phase II have been described. AA evaluations of yield error 
sources will also be conducted during Phase II. The specific 
task descriptions are included in the following subsections. 

6. 2. 4. 3.1 Yield Model Estimation 

The comparative AA evaluations of the LACIE yield estimates versus 
USDA/SRS yield estimates will be conducted throughout the Phase II 
growing season. These evaluations will be made each month by 
state at the five-state (winter-wheat) level, at the four-state 

•v 
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(spring-wheat) level, and at the nine-state level. Relative 
2 

differences , between the LAC IE and USDA/SRS estimates will be 
determined and evaluated throughout the season. This allows 
evaluation of eight winter-wheat and four spring-wheat models 
developed by CCEA, as well as LAC IE adaptations. 

6. 2. 4. 3. 2 Intensive Test Site Yield Estimation 

In addition to the yield model evaluations, AA will examine LACIE 
yield estimates over the ITS 1 s and compare them with yield propor- 
tions obtained from ground observations by USDA/ASCS personnel. 

3 

Relative differences in these estimates are determined and evalu- 
ated as another assessment of yield accuracy. 

6.2. 4.3.3 Other Yield Error Sources 

As required throughout Phase II AA evaluations , special yield 
problems not previously designated as potential yield error 
sources may be identified. As directed by LACIE project manage- 
ment, special yield accuracy problems will be investigated (pro- 
vided resources are available, or as such investigations are 
given special priorities in lieu of some previously described 
yield error source evaluation tasks) . 

6. 2.4.4 Development and Simulation of LACIE Performance Predicuor 
6. 2. 4. 4.1 Model Development 

In order to investigate the generation of error sources within 
LACIE subsystems and to evaluate their effects upon LACIE wheat 
production estimates, an LPP model will be developed. This LPP 
is a Monte Carlo simulation program which simulates the coverage 


2 LACIE yield - USDA/Si?£ yield , „„ 

LACIE yield * 100 * 

3 LACIE yield - USDA/ASCS yield in 
LACIE yield 
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provided by one or two satellites in Sun-synchronous orbits and 
predicts the acquisition of segments as a function of cloud 
cover. Sun angle, and biophase. It also simulates the error in 
wheat production estimates caused by sampling, classification 
(single and multipass) , signature extension, crop calendar, and 
yield errors in the LACIE system. 

The LPP computes the true values, estimated values, true variance, 
and LACIE-estimated variance for wheat acreage, yield, and pro- 
duction estimates at the stratum, zonal, regional, and national 
levels. It also computes true and estimated confidence levels. 

During Phase II, the LPP model initially is used to study the 
advantage of a two-satellite system and to simulate the effect 
on production estimates of sampling error, classification error, 
and yield error. These applications are described in 
sections 6. 2. 4, 4. 2 through 6. 2. 4. 4. 5. 

6.2.4. 4.2 One- Versus Two-Satellite Acquisition Study 

Every allocated sample segment in a biophase must be acquired for 
the LACIE to obtain optimum estimates. The acquisition rate for 
a given biophase is defined as the ratio of the number of segments 
acquired in that biophase to the number of segments allocated. 

The acquisition rate is often less than unity because cloud cover 
prevents some acquisitions. This is particularly true for bio- 
windows of short duration. In such cases it is expected that the 
acquisition rate will be greater with a two-satellite system than 
with a one-satellite system. 

The study described below assesses the improvement in acquisition 
rate expected from a two-satellite system. The study consists of 
Monte Carlo simulations of segment acquisitions in each biophase. 
The acquisition rate for the one- and two-satellite systems will 
be studied for zone, region, and country. Cloud cover will be 


6-26 



simulated using appropriate cloud cover statistics. At least 
10 Monte Carlo trials will be made for each satellite system. 

Each trial will provide an acquisition rate for each biowindow 
so that comparisons of acquisition rates can be made over all 
combinations of biophases. Specifically, the following hypothesis 
will be tested 

Hq : U 2 < u-j_ versus : y 2 > 

where y^ and denote the acquisition rates for one- and two- 
satellite systems, respectively. 

Actual acquisition rates for LACIE Phases I and II will be com- 
pared to the simulation results. 

6. 2. 4. 4.3 Sampling Error Simulation 

Sampling errors are assessed by zone and region, with simulation 
runs assuming no classification error. The extent and scope of 
the simulations depend upon the availability of a proper data 
base; that is, estimates of the within-substratum and year-to- 
year substratum variances. These simulations are performed with 
and without simulated cloud cover so that sampling error with and 
without cloud cover is evaluated. 

Simulations of sampling error are made at substratum, stratum, 
and zonal levels. This input is based on the current ongoing 
effort of estimating within-county variance in the U.S. Great 
Plains. Expected values (i.e., averages) of the acreage esti- 
mates and their corresponding variance estimates will be made by 
10 Monte Carlo repetitions for each case. True acreage and the 
actual variance of an acreage estimate are determined by zone, 
region, and country. Thus, both the acreage estimates and the 
variance estimates are assessed for bias by comparing expected 
values to true values. 
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6. 2. 4. 4. 4 Classification Error Simulation 

Simulation runs are made using the LPP to study the effect of 
classification errors corresponding to single-pass and multipass 
classifications and their combinations on acreage estimates by 
zones and regions. Blind-site data from Phase II will be uti- 
lized to estimate the biases and variances associated with. these 
component errors . 

An evaluation of sampling and classification error components is 
made by comparing estimates with their expected values for the 
different cases of classification error inputs along with sampling 
error inputs. Furthermore , multiple tests will be performed to 
assess the relative significance of the various error components. 

A minimum of 10 replications will be required for each test case. 

Error model 2 analyzes the effect on acreage estimates of omission 
and commission errors, as well as mixed pixel errors (if good 
estimates of variance can be obtained from the blind-site and 
ITS data) . 


6. 2. 4. 4. 5 Yield Error Simulation 

Simulation runs are made to provide an estimate of the relative 
effects of yield as compared with acreage estimation error and 
the respective ultimate effects of each on production. The 
importance of this part of the model will be heavily dependent 
on the availability of historical yield estimation results. 

Since yield modeling has improved considerably in recent years, 
this may involve estimates of how well the present model would 
have performed in previous years. A file of year-to-year biases 
and error variances for different estimation dates for each CRD 
will be required for a meaningful evaluation. At present, the 
date and availability of these data are uncertain. 
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6.3 ACCURACY ASSESSMENT REPORTING 


During LAC IE Phase II operations, AA will provide LAC IE project 
management timely information relative to the accuracy of LACIE 
estimates of wheat acreage, yield, and production and with respect 
to specific problems that may affect these estimates. Three basic 
types of reporting will be utilized: 

a. Special reports and presentations to provide the most recent 
status data on LACIE estimates and to identify potential 
problems as they are disclosed. 

b. AA monthly quick- look reports , which contain rapid evaluations 
of the CMR’s and preliminary comparisons of the LACIE esti- 
mates with USDA/SRS estimates throughout the growing season. 
These reports will provide LACIE project management with com- 
parative status information relative to LACIE products 
throughout Phase II. 

c. Interim and final AA reports , analytical reports throughout 
Phase II, which include a series of interim reports in which 
analyses of early- and late-season winter- and spring-wheat 
LACIE estimates are reported. The interim reports will evolve 
into a final AA report presenting evaluations of LACIE esti- 
mates for the entire Phase II program. 

6.3.1 SPECIAL MANAGEMENT BRIEFINGS AND PRESENTATIONS 

As required to support LACIE reporting functions , special brief- 
ings will be prepared for LACIE project management relative to 
the current status of LACIE operational data development, par- 
ticularly with respect to special problems that could affect the 
accuracy of final LACIE at-harvest estimates of wheat acreage, 
yield, and production in the U.S. Great Plains. 

In addition, AA will support LACIE project management requirements 
to brief NASA, USDA, and NOAA upper- level management on the status 
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of LAC IE outputs and the progress being made toward satisfying the 
90-90 criterion. 

These briefings will be prepared in a format that is consistent 
with other information being presented. 

6.3.2 MONTHLY QUICK-LOOK REPORTS 

The monthly AA quick- look reports are released 7 days after the 
CMR's and contain evaluations of the LACIE estimates presented 
in the CMR's. These evaluations include reviews of results and 
analyses of the internal consistency of the CMR and of the 
statistics presented .in the CMR. In addition, preliminary com- 
parisons will be made with USDA/SRS estimates for the same period 
in the growing season. The purpose of these quick-look reports 
is to provide quick estimates as to the status of LACIE estimates 
and to detect any early anomalies and problems that should be the 
subject matter of further study and analysis by LACIE AA and 
operations personnel. 

These AA monthly quick- look reports will be released after 
approval by the AA manager and the branch chief of the Research, 
Test, and Evaluation Branch (RTEB) of NASA/JSC. 

6.3.3 INTERIM AND FINAL REPORTS 

The Phase II AA final report shall be developed through a series 
of interim reports that provide AA evaluations throughout 
Phase II. Each subsequent interim report includes additional 
AA evaluative data that have become available since the last 
report. In addition, subsequent interim reports will incorporate 
changes that result from analyses and evaluations of previous 
interim reports by LACIE project and technical personnel rela- 
tive to AA. Figure 6-1 illustrates the developmental process 
of the Phase II AA reports, along with the format and basic 
subject matter of each. As can be noted from figure 6-1, the 
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critiques and comments of project technical and management per- 
sonnel on each interim report will be addressed and resolved; 
the basic intent of these comments will be incorporated into the 
succeeding interim report or reports and in the final Phase II 
AA report. 

A detailed outline' to be used in the final Phase II AA report 
is presented in table 6-2. This outline contains the reporting 
elements that are considered necessary for this evaluation report. 

The planned schedule for release of the LAC IE interim AA reports 
and the final Phase II AA report is as follows; 

e First interim report — May 15, 1976 

• Second interim report — August 15, 1976 
a Third interim report — November 15, 1976 

• Final interim report — March 1, 1977 
® Final Phase II AA report — 1977 

The AA interim reports will require approval by the AA manager 
and the Chief of the RTEB of NASA/JSC prior to their release for 
LACIE project review and evaluation. 

The final LACIE Phase II AA report will require review and 
approval of the following persons prior to its release for 
distribution: 

J. L. -Dragg, Chief, Applications Systems Verification 
Branch, NASA/ JSC 

R. B. Erb, Chief, LACIE Project Office, NASA/ JSC 

J. D. Erickson, Chief, RTEB, NASA/JSC 

F. G. Hall, LACIE project scientist, NASA/JSC 

J. Hill, assistant deputy project manager for LACIE, 

NOAA 
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R. B. MacDonald, Chief, Earth Observations Division, 
NASA/JSC 


J. Murphy, assistant deputy project manager for LACIE , 
US DA 

D. E. Pitts, manager, LACIE AA, NASA/JSC 
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TABLE 6-1.- YIELD 

MODEL 

DATA 

REQUIREMENTS SCHEDULE 

FOR 


ACCURACY ASSESSMENT 




Yield model 

Mar. 

Apr 

. May 

June 

July 

Aug . 

Winter wheat: 







Tex. 

X 

X 


X 

X 

X 

Okla. 

X 

X 


X 

X 

X 

Okla./Tex. 

Panhandle 

X 

X 

X 

X 

X 


Kans . 

X 

X 


X 

X 

X 

Colo. 


X 

X 

X 

X 

X 

Nebr . 



X 

X 

X 

X 

N. Dak./S. Dak. 


X 

X 


X 

X 

Mont. 


X 

X 

X 

X 

X 

Spring wheat: 







S. Dak. 


X 

X 


X 

X 

Red River Valley 
(Okla. /Tex. ) 


X 

X 


X 

X 

N. Dak. 


X 

X 

X 

X 

X 

Mont . 


X 


X 

X 

X 


NOTE: Months in which data are not required indicate that 


no meteorological data were input into that particu- 
lar model from the previous month; therefore, yield 
predictions would not change. 

The associated yield contributions of each coeffi- 
cient, of the F- value for each variable, and of the 
correlation coefficient for each yield input factor 
are required once a year. 
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TABLE 6-2.- DETAILED OUTLINE OF THE PHASE II ACCURACY 


ASSESSMENT REPORT 


1 . INTRODUCTION 

1.1 OBJECTIVES OF PHASE I AND PHASE II 

• To determine how well LACIE is performing in the yardstick 
region 

• To determine causes of LACIE error and to quantify, if 
possible 

• To make recommendations as to potential procedures or 
methods for reducing the error 

1.2 GENERAL METHODOLOGY 

A general discussion of 'the methods to be used by AA in: 

• Data gathering 

• Analysis and evaluation 

• Reporting 

2 . SUMMARY 

A general summary of significant results in the time period 
covered will be made in the following subsections: 

2.1 EVALUATION OF PRODUCTION ESTIMATES 

2.1.1 PffikSE I MONTHLY PRODUCTION ESTIMATE AND REWORK 

2.1.2 PHASE II MONTHLY PRODUCTION ESTIMATE 

2.1.3 COMPARISONS OF PHASE I AND II PRODUCTION ESTIMATES 
BY MONTH 

2.2 EVALUATION OF ACREAGE ESTIMATES 

2.2.1 PHASE I MONTHLY ACREAGE ESTIMATE AND REWORK 

2.2.2 PHASE II MONTHLY ACREAGE ESTIMATE 

2.2.3 COMPARISONS OF PHASE I AND II ACREAGE ESTIMATES BY 
MONTH 
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TABLE 6-2 .— Continued. 


2.3 EVALUATION OF YIELD ESTIMATES 

2.3.1 PHASE I MONTHLY YIELD ESTIMATE 

2.3.2 PHASE II MONTHLY YIELD ESTIMATE 

2.3.3 COMPARISONS OF PHASE I AND II YIELD ESTIMATES BY 
MONTH 

3. ACCURACY ASSESSMENT METHODOLOGY 

A detailed technical discussion of the methods used and the 
rationale for their usage will be presented, including dis- 
cussions of: 

3.1 OPERATIONAL MODELS 

The yield estimation model and the production, acreage, and 
yield aggregation models used in LAC IE will be discussed 
briefly and referenced. 

3.2 GENERAL STATISTICAL MODELS 

The specific formulation of the AA statistical models that 
are the basis for subsequent error budget evaluations will 
be presented, including: 

3.2.1 PRODUCTION MODEL 

3.2.2 ACREAGE MODEL 

3. 2. 2.1 Classification Error 

3. 2. 2. 2 Sampling Error 

3.2.3 YIELD MODEL 

3.3 ERROR BUDGET 

_The statistical methodology that results from the general AA 
statistical models described in section 3.2 will be utilized 
to quantify the errors in LACIE estimates and to determine if 
these are acceptable in order for LACIE production estimates 
to meet the 90-90 criterion for both Phases I and II. This 
section will include a discussion of how this error is dis- 
tributed over the acreage and yield components, along with 
the specific formulation of the statistical estimates of the 
production error components broken down as follows. 
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TABLE 6-2.— Continued. 


3.3.1 ACREAGE ERROR 

3. 3. 1.1 Classification Error 

3. 3. 1.2 Sampling Error 

3.3.2 YIELD ERROR 

3.4 COMPARATIVE EVALUATIONS OF LAC IE ESTIMATES WITH 
REFERENCE DATA (USDA/SRS and blind-site ground truth) 

3.4.1 LAC IE VERSUS USDA/SRS ESTIMATES 

Comparative evaluations of LACIE estimates will be made with 
USDA/SRS estimates of production, acreage, and yield at the 
state, five-state, and nine-state levels within the U.S. 
Great Plains. Relative differences will be calculated and 
examined to determine any problem areas that may exist. 

3.4.2 LACIE WHEAT/SMALL-GRAIN PROPORTIONS VERSUS GROUND 
OBSERVATIONS OVER THE U.S. GREAT PLAINS BLIND SITES 

Comparative evaluations will be made of LACIE proportion 
estimates of wheat/small grains with ground truth obtained 
from observations by USDA/ASCS personnel over the U.S. Great 
Plains blind sites. 

3.5 LACIE PERFORMANCE PREDICTOR ERROR SOURCE SIMULATIONS 
AND EVALUATIONS 

The usage of the LPP model to evaluate error sources and 
their effects upon LACIE production estimates will be dis- 
cussed, along with simulations of acquisitions of LACIE 
segments utilizing one or two satellites to determine 
potential advantages . 

4. EVALUATIONS OF LACIE PRODUCTION ESTIMATES 

AA evaluations utilizing the methodology described in sec- 
tion 3 will be reported. LACIE and USDA/SRS production 
estimates will be compared and examined to determine if 
the 90-90 criterion is being met. This will be done at 
the state, five-state, and nine-state levels for both 
Phases I and II. 
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TABLE 6-2.— Concluded. 


5. EVALUATIONS OF LAC IE ACREAGE ESTIMATES 

LAC IE Phase I and II acreage estimates will be evaluated 
through comparisons (1) between wheat proportion estimates 
(P) and ground truth (P ) over blind sites and ITS 1 s from 

the U.S. Great Plains and (2) between LACIE acreage estimates 
and USDA/SRS estimates by month, by state, and by region. 

Error budget checks of LACIE estimates will also be reported. 

Specific acreage error source investigations will also be 
presented. 

6. EVALUATIONS OF LACIE YIELD ESTIMATES 

LACIE yield estimates for Phases I and II will be compared 
with comparable USDA/SRS values by month at the CRD, state, 
and regional levels. Further investigations of the yield 
model will be conducted, along with specif c investigations 
of potential error sources in LACIE yield estimates. 

7. DETAILED FINDINGS OF LACIE ERROR SOURCE INVESTIGATIONS 
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APPENDIX B 


LAC IE BLIND AND INTENSIVE TEST SITES 


The LAC IE ITS 'and blind-site selections are given in the follow- 
ing tables. 


Table 


B-l 

B-2 

B-3 


B-4 


Sites 

LACIE Phase II U.S. ITS ' s 

LACIE Phase II CANADIAN ITS 1 s 

Early-season blind-site selection from the 
U.S. southern Great Plains 

At-harvest blind-site selection from the 
U.S. Great Plains 
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TABLE B-l LAC IE PHASE II U.S. INTENSIVE TEST SITES 


Segment 

State 

County 

Center coordinates 

Site size 

Wheat 

Acquired 

as 

Lat., N 

Long., W 

N. mi. 

Km 

type 

(a) 

1961 

Kans. 

Morton 

37°16 ' 00" 

101° 54 1 00" 

5x6 

9xll 

WW 

ww 

X962 

Kans. 

Saline 

38°41' 48" 

97°28 1 24" 

3x3 

5. 5x5. 5 

ww 

ww 

X9 63 

Kans . 

Rice 

38°17' 00" 

9 8° 12' 42" 

3x3 

5. 5x5. 5 

WW 

ww 

X964 

Kans . 

Ellis 

38° 50 ' 06" 

99° 13' 00" 

3x3 

5. 5x5. 5 

ww 

ww 

X988 

Kans . 

Finney 

38°10' 12" 

100°43'12" 

5x6 

9xll 

ww 

ww 

X965 

N. Dak. 

Burke 

48°53'12" 

102°10' 00" 

5x6 

9x11 


sw 

X966 

H. Dak. 

Williams 

48°19 1 12" 

103°24' 42" 

5x6 

9xll 


sw 

1967 

N. Dak. 

Divide 

48° 53 ’36" 

103°10* 54" 

2 x 10 

3.7x18.5 

sw 

sw 

1968 

Mont. 

Glacier 

48 o 37'30" 

112°33 1 24“ 

2x10 

3.7x18.5 

S&WW 

sw 

1969 

Mont. 

Toole 

48°53 ' 00" 

111°46'36" 

2x10 

3.7x18.5 

S&WW 

sw 

1970 

Mont. 

Liberty 

48° 44 1 00" 

110°51'00" 

2x10 

3.7x18.5 

S&WW 

sw 

1971 

Mont. 

Hill 

48°42 ' 00" 

109°55' 00" 

' 2x6 

3.7x11 

S&WW 

sw 

1972 

Wash. 

Whitman 1 

46° 54 1 36" 

117°15' 30" 

3x3 

5. 5x5. 5 

S&WW 

ww 

1973 

Wash. 

Whitman 2 

46° 50 ' 24" 

117°48' 18" 

3x3 

5. 5x5. 5 

S&WW 

ww 

1974 

Wash. 

Whitman 3 

47° 08 ' 00" 

117°26'18" 

3x3 

5. 5x5. 5 

S&WW 

ww 

1975 

Idaho 

Oneida 

42° 04' 30" 

112° 29 ' 30" 

3x3 

5. 5x5. 5 

S&WW 

ww 

1976 

Idaho 

Franklin 

42°08' 00" 

111°58 , 00" 

3x3 

5. 5x5. 5 

S&WW 

ww 

1977 

Idaho 

Bannock 

42 6 56 1 30" 

112° 25' 50" 

3x3 

5. 5x5. 5 

S&WW 

ww 

1978 

Tex. 

Randall 

35°09 1 30" 

102° 04’ 24" 

3x3 

5. 5x5. 5 

ww 

ww 

1979 

Tex. 

Deaf Smith 

34°52' 12" 

102° 22 ' 18" 

3x3 

5. 5x5. 5 

ww 

ww 

1980 

Tex. 

Oldham 

35° 15 1 00" 

102°32 ' 00" 

3x3 

5. 5x5. 5 

ww 

ww 

1981 

Ind. 

Shelby 

39°27’ 36" 

85°47'12" 

3x3 

5. 5x5. 5 

ww 

ww 

1982 

Ind. 

Madison 

40°13 ' 30" 

85°37'50" 

3x3 

5.5x5. 5 

ww 

ww 

1983 

Ind. 

Boone 

40°05 ' 42" 

86°33' 90" 

3x3 


ww 

ww 

1687 

S. Dak. 

Hand 1 

44°35 ’ 00" 

98°58’ 00" 

5x6 

9x11 

S&WW 

sw 

1986 

S . Dak . 

Hand 2 

44°21 ' 00“ 

98°45'06" 

5x6 

9xll 

S&WW 

sw 

19 87 

Minn, 

Polk 

47°49' 00" 

96°41*00“ 

5x6 

9xll 

sw 

sw 


a As indicated by ground truth: S&WW = spring and winter wheat. 
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TABLE B-2.- LAC IE PHASE II CANADIAN INTENSIVE TEST SITES 


Segment 

Province 

County 

Center coordinates 

Site 

size 

Wheat 

type 

Lat. , N 

Long. , W 

S . mi. 

Km 

19 58 

Saskatchewan 

Melfort 

52° 48 ' 

104° 44 ' 

2x10 

3.2X16 

SW 

1959 

British Columbia 

Dawson Creek 

55° 4 8 1 

120°12 ' 

2X10 

3.2X16 

SW 

1984 

Saskatchewan 

Delisle 

51°55 1 

107°28 ’ 

2x10 

3.2x16 

SW 

1985 

Saskatchewan 

Swift Current 

50°19 ' 

. 107°53 ' 

2X10 

3.2x16 

SW 

1989 

Alberta 

Lethbridge (Raymond) 

49°30 1 

112° 4 8 ' 

2xlo 

3.2X16 

SW 

1990 

Manitoba 

Stony Mountain 

50°04 ' 

97°21 ' 

2x10 

3.2x16 

SW 

1991 

Manitoba 

Starbuck 

49° 47 ' 

97° 29 ’ 

2X10 

3.2x16 

SW 

1992 

Alberta 

Olds 

51° 54 1 

113°32 ' 

2xio 

3.2X16 

SW 

1994 

Alberta 

Ft. Saskatchewan 

53°38 1 

113°07 ' 

2X10 

3.2X16 

SW 

1995 

Manitoba 

Altona 

49° 12 1 

97° 38 1 

1X5 

1.6X8 

SW 




TABLE B-3 EARLY-SEASON BLIND-SITE SELECTION FROM THE 
U.S. SOUTHERN GREAT PLAINS 


State 

Number of 
group 1 
segments 

Number of 
group 2 
segments 

Number of 
code 40 
segments 

Colo. 

3 

1 

2 

Kans . 

10 

5 


Nebr . 

4 

1 


Okla . 

4 

6 

1 

Tex. 

10 

3 

1 

Total 

31 

16 

4 


Total 
number of 
segments 

Expected 
number of 
blind-site 
segments 

Expected 
date of 
ground- trut] 
collection 

6 

5 

Late March 

15 

10 

Mid-March 

5 

5 

Mid-March 

11 

10 

Early March 

14 

10 

Early March 

51 

40 




TABLE B-4 AT- HARVEST BLIND-SITE SELECTION FROM THE U.S. GREAT PLAINS 


State 

Number of 
group 1 
segments 

Number of 
group 2 
segments 

Total 
number of 
segments 

Number of 
segments 
processed 
in 1975 

Expected 
number of 
blind-site 
1 segments 

Expected 
date of 
ground- truth 
' collection 

Colo . 

27 

5 

32 

24 

12 

July 

Kans . 

73 

11 

84 

50 

25 

June 

Minn. 

8 

5 

13 

9 

5 

August 

Mont. 

57 

3 

60 

39 

20 

July 

Nebr . 

28 

7 

35 

23 

12 

June 

N. Dak. 

65 

0 

65 

42 

20 

August 

Okla. 

33 

7 

40 

29 

15 

May 

S . Dak . 

30 

3 

33 

23 

12 

July 

Tex. 

38 

11 

49 

28 

15 

May 

Total 

359 

52 

411 

267 

136 
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SYMBOLS 



B 


C 

a 


COV 

CV 


E 



E 



n 

N 

Pr 


V 

w 




Y 


y: 


X 


true wheat acreage in the j th CRD. 
bias . 

blind-site wheat proportions from ground truth. 

percentage point of the standard normal distribution at 
the a-level of significance. 

covariance, 

coefficient of variation, 
estimate. 

expected value of e 

r 

expected value of s 

Sj 

implied statement, 
number of samples. 

total number of CRD ' s in the zone. 

I probability. 

variance. 

true wheat production. 

USDA/SRS value for wheat production. 

production estimate for the j th CRD . 

county wheat proportions from the 1969 Agricultural 
Census . 

true wheat yield. 

LACIE wheat proportion estimate from the i th blind site. 
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CN [g 



true wheat yield in the jth CRD. 


W - (W + B) 



level of significance. 

error resulting from classification. 

error resulting from sampling. 

production estimation error. 

yield estimation error. 

true variance of classification error. 

true variance of classification error in the jth CRD. 

true variance of sampling error. 

true variance of sampling error in the jth CRD. 

true variance of production estimation error in the 
jth CRD. 

true variance of yield estimation error in the jth CRD. 
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1 . INTRODUCTION 


This appendix contains a description of methods to be used in 
(i) determining if the LACIE production estimate is meeting the 
90-90 criterion and (ii) estimating various error components. 

In section 3, it is shown that the 90-90 criterion will be 
satisfied if the CV for the production estimate at the U.S. 

Great Plains level, CV ( W) , is less than a certain upper bound. 

This is the method used by AA to determine if the 90-90 criterion 
is being met. The models and assumptions used in the calculation 
of CV (W) are described in section 2 of this appendix. 

The methods of estimating error components are set out in sec- 
tion 4, and the effects of various error components on the produc- 
tion estimate are described in section 5. 

The general statistical models proposed in this report express 
merely the functional relationship among the estimates , the 
true values of the estimates, and the error components of the 
estimates. The detailed description of the LACIE operational 
models and methodologies , such as acreage aggregation and yield 
prediction, may be obtained from the CAS and YES requirements 
documents (refs. 1,2). 

The assumptions made in the models are those which have been 
made in the LACIE Phase II operational system. The models are 
subject to modification to incorporate the new algorithms when 
they become officially available for the LACIE operational 
system. 
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2. GENERAL STATISTICAL MODELS 


In this section, the statistical models for the acreage and 
yield estimates are presented, and these estimates are then 
incorporated into a model for the production estimate. The 
variance (V) and CV of e'ach estimate are derived in a general 
form based on each formulated model and the corresponding 
assumptions . 


2 . 1 ACREAGE 


The statistical model for the estimate of wheat acreage is formu- 
lated as follows. Let N be the total number of CRD 1 s in the zone 
under investigation and let iL be the estimate of the true wheat 
acreage, A , in the j th CRD. Then the general statistical model 
for the acreage estimate in the j th CRD is given by 


A . = A + e 
3 3 c j 


+ e, 


( 1 ) 


for j = 1,2,***,N, where e and £ are the errors resulting 

3 S J 

from classification and sampling, respectively. The following 
assumptions are made concerning these errors. 


1 - £ c is a random variable with mean zero and unknown variance 

-- iL ~ 
cr^ ; that is , 

j _____ 

E ( e Cj ) = 0 and V ( e Cj ) “ a c. ,2) 


where E/£ 


(S) 


= expected value of £ 


2. £ and £ are uncorrelated, i ^ j, so that COV/£ ,£ 1 

j C i V C j C i, 

where COV/e ,e \ is the covariance of e and e 

c . c . ) c . c . 

\ 3 i/ 31 


0, 
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3. e is also a random variable with mean zero and unknown 
j . ' 2 

variance o ; that is , 
s 




S/£ \ = 0 

and 

V/ E - \ = al 

(3) 



\ 3 I 


{ s l) s j 


4 . £ 

s. and 
3 

e are uncorrelated, 

O • 

l 

i ^ j, so that COV /e 

l j 

'V 

Thus , 

E (A.) 

A 

= A (=*> A . is an 
3 3 

unbiased estimator of A ^ ) , 

• 



V(A.) = a 2 + 

3 3 

2 

a 

for j = 1 , 2 , • * • ,N 

(4) 



c j 



and 


COV (A . ,A . ) = 
J 

0 

for all i 7 ^ j 

(5) 


Consequently, the CV of the acreage estimate, CV (A j ) , for the 
j th CRD is given by 

2 * 

Cv (A . ) = — 

3 A Z 


a 

c . 
1 

A? 

3 


a 


A? 


c . 

3 


a 


a -fa 
c . s . 

3 3 


-cv <V +72 


a + a 

Cj Sj 


a, CV (a . ) 


= cv 2 (a. jc) + cv 2 (A. Is) 


( 6 ) 


where 

A j 

CV(Ajjc) = the CV of the acreage estimate resulting- from 
classification error. 


CV(Aj|s) = the CV of the acreage estimate resulting from 
sampling error. 
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Since the acreage estimate A for zone is 


N 


a = a. 
Lj j 


( 7 ) 


i=l 


under the assumption in equation (5) , the variance of A is given by 

N 

V (A) = V(A ) 
i=l 


o 2 

= a + a 
c s 


( 8 ) 


N N 

. 2A2 2 J— v 2 

where 0 = \ a and a = \ a 

C / J C . S / J s . 

3=1 3 j=l 1 


Thus , 


CV 2 (A) 


2 + 

A A 


= CV 2 (A I c) + CV 2 (£|s) 


(9) 


2.2 YIELD 

The LACIE Phase II operational yield model was formulated on a 
regional basis. The Phase II operational procedure implements 
this model at the district and zone levels to generate the 
yield estimates and variances of yield estimates for the dis- 
trict and zone, respectively. 

The following statistical yield model is formulated according 
to the LACIE Phase II procedure. It reflects only the relation- 
ship between the estimate and the' true value and the necessary 

A 

assumptions in the model. Let Y^ be the yield estimate for the 
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( 10 ) 


true yield Y of the ith CRD. The model may be written as 
0 

Y. = Y. + 

3 3 Y, 


where e Y i s a yield error predominantly from equational error. 

j 

The assumptions for the model are given as follows. 

1. s j is a random variable with mean zero and unknown variance 
2 

(7^ ; that is , 



and 

V (^) = 

2. and e Y are uncorrelated, i ^ j, so that 

i 3 


( 11 ) 


( 12 ) 


COV|'e Y> ,e Y ^ = 0 

Thus , 

E ‘V = Y. 

V (Y . ) = a Y 

3 j 

and Y^ and Y ^ , i ^ j , are uncorrelated. 


(13) 


(14) 

(15) 


2.3 PRODUCTION 

/N 

Let Wj be the estimate of the true production for the j th 
CRD. The statistical model for the production estimate is 
formulated as W. = W. + e w for i = 1,2, •♦*,N. The production 

J 3 j 

estimation error may be shown as a function of sampling, clas- 

. . ' j 

sification, and yield errors. In particular, by derivation, 
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w. 

3 


A.Y . 

3 3 


h 


+ + £ 

C . S 

3 3 


j)( s J + e h) 


A.Y. + e Y. + s Y. + A.e + £ c £ y + £ s £ Y (16) 
3 3 c. 3 s. ] 3 * 3 C D s j X D 


It follows that 


£r? = £ _ Y . + E Y. + A. £ + / E + £ \£ v 

W. c . n S-l i Y . ( c . s.Y. 

3 3 J 3 J J 3 l 3 " 




(17) 


The assumption which is made for deriving the variance of produc- 
tion in LACIE Phase II is that 


E 


' £ + £ \ £, r 
c . s.Y 
3 3) 3 


pi] ‘ Th + sn%) 


which implies that 


✓s /\ 


COV(A. ,Yj) = 0 


(IB) 


A further assumption, which is not immediately necessary and 
will be recalled and discussed when used, is that the production 
error £^ is normally distributed with mean zero and unknown 
j 2 

variance a r7 ; that is, 

W . 


£ w * N (°' a w j ^ 

Under assumptions in equations (2) , (3) , (11) , and (18) , the 

following results are obtained from equation (17) . 


E 


(%) 


= 0 


That is. 


E (W • ) = W. 
3 3 


( 20 ) 
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By taking the variances of both sides of equation (16) , one 

obtains: V (W . ) = Y 2 V (A . ) + A?V (Y . ) + V(A.)V(Y.). It is note- 

D 3 3 3 3 3 3 

worthy that the unbiased estimator for V(W*) / V (W ), is given 

by 


/s /N /v 


V(W.) = Y?V(A.) + A 2 V(Y.) - V (A ) V (Y . ) 
3 3 3 3 3 3 3 


( 21 ) 


where Y . , A . , V (Y . ) , and V (A . ) are the estimates of Y . , A . , V (Y . ) , 
ysD33 3 3 3 3 

and V(Aj), respectively; and since yield variance is under- 
estimated at the district level, the production variance obtained 
from equation (21) is underestimated also in the LACIE Phase II 
system. 

The CV of production at the nine-state level is given by 


CV(W) = 


N 


-, 1/2 


E V( V 

l.i=i 


w 


N 


W = 2 W - 
j“l 


( 22 ) 


3 . ERROR BUDGET 


This section describes the method of determining the upper 
bound on the CV of production allowed by the 90-90 criterion. 
The 90-90 criterion can be expressed by the following proba- 
bility statement. 
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Pr [ jw - W| < 0.1W] > 0.90 
=> Pr [-0 . 1W < W - W < 0.1W] > 0.9 0 
=*> Pr [-0 . 1W - B < W - (W + B) < 0.1W - B] > 0.90 


Pr 


-0.1W - B < W - (W + B) Q.1W - B 

(JSX c /n — fi ^ 

w w °w 


> 0.90 


( 23 ) 


Where the bias B — E(W) - W and cr^ = [V(W)]"^^^. Under the assump- 
tion stated in equation (19) , equation (23) becomes 


Pr 


-0.1W - B 
W + B 

CV (W) 


< z < 


Pr 


- 0.1 - 


0. 9B 
B t W 


0.1W - B 
W t B 

CV(W) 

0.1 - 


> 0.90 

1.1B 


CV(W) 


< z < 


B + W 


CV(W) 


> 0.90 


(24) 


(25) 


a 


'Where CV(W) = 


W 


E (W) 


W - (W + B) . 

and Z = is the relative error of 

fT A 

u w 


the proportion estimate (relative to a~) having standard normal 

w 

distribution; that is, Z a, N(0,1). 


NOTE : Since a large number of samples are utilized for the LAC IE 

estimates , the distribution of the relative error approxi- 
mates standard normal distribution. Thus, the assumption of 
normality for e w may be eliminated in the LAC IE study. 

j 

The assumption of equation (19) is made for developing 
equation (25) without the consideration of sample sizes. 


A 

If W is an unbiased estimator of W, then B = 0 and equation (25) 
reduces to 


or 


Pr 


~- 0.1 

_CV(W) 


< 


z < 


o.i ~ 

CV(W)_ 


> 


0.90 


Pr 



< 


0.1 ~ 

CV(W)_ 


> 0.90 


(26) 
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Thus, it follows that — = 1.645, which implies that 

CV(W) 

CV (W) = 0.06 is the maximum CV of the production estimate that 
can be tolerated for the production estimate if the 90-90 cri- 
terion is to be satisfied. 


Since only one observation for W occurs at the nine-state level 
in LACIE Phase II, the bias B cannot be accurately estimated. 
However, a confidence interval might be computed to determine 
the upper and lower limits for B. If it is assumed that the 
USDA/SRS value for wheat production, Wg , is the true value, then 


and 


- W - (W + B) 

Z = ~ N(0,D 

W 


Pr 



W - (W + B) 

< y < 

- etc - 

W 



1 - a 


(27) 


where a is the known significance level, and C a is the percentage 
point of the standard normal distribution at the a-level. 


For the given a, W, Wg , and the confidence limits for the 

bias can be obtained directly from equation (27) . 


4. ERROR COMPONENT ESTIMATION 


The formulas for estimating the various error components for the 
LACIE Phase II AA are presented in this section. 

4.1 ACREAGE ESTIMATION ERROR COMPONENTS 

In the acreage model, equation (1), the two major error com- 
ponents are derived from classification and sampling. If 

. } represents the blind-site wheat proportions from ground 

i=l 
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truth, and {X.} n represents the county wheat proportions from 

1 1=1 2 
the 1969 Agricultural Census, then the classification error 

is estimated by 


-2 

a 

c 


i n 

si 

i=l 


(Y! 

l 



(28) 


where Y^ is the LACIE wheat proportion estimate from the i th 

blind site and n is the number of samples. The sampling error 
2 . 

a is estimated by regressing b. on X- to obtain 


s 2 

s 



i=l 

n - 2 


(29) 


A 

where b^ is the predicted value of b^ from the regression. 


4.2 PRODUCTION ESTIMATION ERROR COMPONENTS 

Since production is the product of acreage and yield, the acreage 
and the yield errors comprise the production error. The formulas 
for estimating these error components are given at the zone or 
higher level in this section. 


4.2.1 ACREAGE 

The acreage error (variance) for the zone or higher level was 
shown in equation (8) , and assumptions were made and discussed 
in section 2.1. 


4.2.2 YIELD 

The yield error is more complicated and must be expressed in 
terms of variances for acreage and production estimates. The 
yield variance estimate for the zone or higher level, as given 
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xn the CAS software requirements document (ref. 3) , is as 
follows. 



where 

A 

Y = yield estimate for the area. 

A 

Wf = production estimate for the 1th CRD. 

A. A. 

V(W ± ) = estimated variance of W^. 

A 

A^ = acreage estimate for the 1th CRD. 

V(A^) = estimated variance of A^. 

A 

Y^ = yield estimate for the 1th CRD. 

S = number of CRD ' s within the area. 

An approximate formula for V(Y) can be derxved in the following 
manner . 


Let the statistical model for yield estimation at the zone or 
higher level be 

Y = Y + with E ( £y ) = 0 (30) 

A 

where Y is the estimate of the true yield Y at this level, then 

V (Y) = V(e y ) = e(s 2 ) 


= E[ (Y - Y) 2 ] 



( 31 ) 
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If the sample size is large, A should not differ greatly from A. 
The approximation consists of replacing A by A in the demoninator 
of equation (32) . This gives 

V (Y) S ~E[W - AY] 2 (32) 

A 


Since it was assumed E (W) = W, E (A) = A, and COV(A,Y) 
equation (32) can be written as follows. 

V (Y) = -^E(W + A Y^ - 2WAY) 

A 


= 0, 


= -^-jv(W) + Y 2 V(A) - 2Y [COV (WA) ] ]■ 


(33) 


Since 


COV (WA) = E (WA) - E (W) E (A) 


/■v 9 9 

= E(A )Y - Ax 
= [V (A) + E 2 (A) ] Y - A 2 Y 


= YV (A) 

equation (33) can be written as 


V (Y) = — ij[V (W) - Y 2 V(A)] 


(34) 


|Y 2 V(W) V (A) Y 2 


2 2 
AY 


A 


= Y‘ 


V(W) V (A) 


W 


O O g 

Consequently, this gives Cv (Y) = CV (W) - Cv (A) or 
cv 2 (w) = CV 2 (Y) + CV 2 (A) . 


(35) 


The estimated variance for the yield estimate, V(Y), may be 
obtained by replacing Y, W, A, V(W), and V(A) in equation (35) 


C-15 



with the estimators Y, W, A, V(W), and V(A), 
This gives 


A A 

V (Y) 




V<A) 


respectively . 


G36) 


Values for V(W), W, V (A) , and A at the zonal (state) level are 
given in the CMR's. 


5. EFFECT OF ERRORS IN ACREAGE AND YIELD ON THE 
VARIANCE OF THE PRODUCTION ESTIMATE 


Although the production error components are the acreage and 
yield errors , the production error at the zone or higher level 
will not be computed directly from these component errors at 
the same level. The effect of these errors on the production 
error will be determined by omitting the corresponding terms in 
the formulas for the variance of the production estimate given 
in section 2.3. 


1. Production variance without acreage error: The variance 

is estimated as in equation (21) , except that the first term 
is omitted; that is. 


^ » /v v*' 

v (w) =2^ 

j 


A . V (Y . ) 
3 3 


V(A j )V(Y j ) 


(37) 


2. Production variance without yield error: It is estimated as 

in equation (21) , except that the second term is omitted; 
that is , 


v (w) = Z, 
j l 


aOa a a a a a 

Y . V (A . ) - V (A . ) V (Y . ) 
3 3 3 3 


(38) 
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Production variance v/ithout sampling error: Equation (21) 

is used to estimate this variance, except that 


V (A j ) is replaced by 


-2 

0 


— o ~V (A . ) ; that is , 

0 + a J 

c s 


S \ 9 1 M 

'i« = Z 
j 


A A A A 


^ p A 0 A A aOa a 

jijVttj) + v (x a> - JT^2 v( V v( V 


^2 , A 
a + a 
c s 


0 +0 
c s 


(40) 

4. Production variance without classification error: The vari- 

ance is estimated as in equation (21), except that V (A . ) is 

a 2 

replaced by x- s — ?rV (A ); that is, 

^ Z. "I 
0+0 J 
C _ S 


^ a v — ^ 

V 2 (W) =2] 


y\2 /\ 2 

A 9 A A A 2 A A 

~ 2 - - .2 Y V (A i } + A 1 V < Y i > * 

0+0 J J J 0“+0 
c s c s 


A A /\ /\ 


2 . 


(41) 


The CV's corresponding to the variances in equarions (37) through 
(41) are given by 


CV L (W) = 


r- ~ l 1/2 
V T (W) 7 

Jj 


for 


W 


L = 1,2, 3, 4 


(42) 


If the reduction in the production CV obtained by omitting the 
acreage error is greater than that obtained by omitting the 
yield error [that is, if CV^ (W) < CV 2 (W) ] , this implies that 
the acreage estimate contributes mbre error to CV(W) than the 
yield estimate. A similar comparison can be made in assessing 
the contributions of the classification and sampling errors to 
CV(W) . 
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INSTRUCTIONS 

FOR 

LAC IE SEGMENT INVENTORY 
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I. Introduction 
A. Background 

The LACIE (Large Area Crop Inventory Experiment) is 
an interagency experiment in the use of Landsat (formerly 
called Earth Resources Technology Satellite) and meteoro- 
logical data to identify and inventory crop production. 
Participating agencies include the Department of Agriculture, 
the National Aeronautics and Space Administration, and the 
National Oceanic and Atmospheric Administration. Within 
the Department of Agriculture, participating agencies are 
the Agricultural Stabilization and Conservation Service, 
Economic Research Service, Foreign Agricultural Service, 
Agricultural Research Service, Soil Conservation Service, 
and Statistical Reporting Service. The overall general 
objectives of the LACIE are to determine utility and cost 
effectiveness of satellite and surface derived data sources 
to monitor large area crop (wheat) production and assess 
the impact of agricultural and meteorological conditions 
on production estimates. The utility of the information 
produced will be evaluated on the basis of its objectivity, 
timeliness, accuracy, and its expected value for policy 
and program decision making. 

LACIE reports are based on data extracted from 5x6 
mile segments that have been randomly placed throughout the 
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wheat producing region of the United States. In order to determine 
our accuracy, it is necessary that we know what is actually in our sample 
segment. The information requested for the segment that has been identified 
and forwarded to you is essential for a successful evaluation of the 
project. The enclosed color prints have been obtained only over the 
selected site in your county to support ground data collection. 

B. Authority 

The USDA LACIE Project Manager has requested that the Agricultural 
Stabilization and Conservation Service provide this function and they 
have accepted the assignment. You should have already gotten an authoriza- 
tion from your State office concerning this task. If you have not, you 
should contact them at once. 

C. Requirements of the ASCS County Office 

You are being asked to do the following: 

1. Review the set of instructions. 

2. Visit the segment location and identify the land uses, 

' even if the segment falls outside your county. 

3. Check over your v/ork and return the completed inventory 
as soon as possible. 

II. Data Collection Procedures 
A. Supplies 

1. Color infrared print or prints. 

t 

2. Mylar overlay. 

3. Topographic map witn segment location. 
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4. Standard crop key. 

5. Crop stage development key. 

6. Evaluation form. 

7. Return post card and return mailing tube label. 

B. In some cases, all of the segment will not be covered by the 
photo. Complete the survey for that portion outlined on the photo. 

C. Procedures 

1. You are required to identify all fields within the segment 
boundaries using codes as indicated on the attached crop 
key (see attached LACIE segment for classification). 

2. Use ball point pen for all coding directly on the mylar. 

3. The photos are provided as a base for field pattern and 
references. 

a. All field identification should be based on actual 
ground conditions on the day that you visit the segment. 

b. If there are any differences between the photo and the 
ground, then footnote each field that is different and 
explain on evaluation form. 

c. If any fields have been harvested at the time of your 

visit, place a /H after the crop code. 

d. If any fields have not been harvested at the time of 
your visit, and from your observations appear to be 

abandoned, place a / A after the crop code, footnote, 

and explain. 

4. Use the evaluation form for all comments on any unusual 
crop condition or practice (irregular, replanting, drought, 
etc.). 
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5. "If there are any crops in the segment for which there is no 

code, select an unused symbol and indicate its meaning on 
the evaluation form. 

6. Assess the average wheat crop stages while completing the 
segment inventory and enter it on the evaluation form upon 
completion. 

III. JSC Contact 

A. If there are any problems, contact the person listed below. 

B. Review procedures and crop key before going into the field and 

contact the Johnson Space Center if there are any questions. 

Bobby E. Spiers, TF4 
U.S. Department of Agri culture/ ASCS 
NASA - Johnson Space Center 
Large Area Crop Inventory Experiment 
Houston, TX 77058 

Phone: Commercial - A/C 713-483-4623 

FTS - 525-4623 

IV. Due Date and flailing Procedures 

A. Upon receipt of data from the Johnson Space Center, complete the 
enclosed post card and return it to JSC. 

B. Field information should be collected within 10 days after 
receipt of material by your office, if at all possible. 

1. Upon completion of field survey, fill out evaluation form 
and return with photos. 

2. Return all material (maps and photos) in the same mailing 
tube you' received data in, using the provided return label. 

C. Thank you for your cooperation and effort -in assisting LACIE 
in this vital area of the experiment. 
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ORIGINAL PAGti 
OF POOS QUALITY 


SAMPLE SEGMENT 


With Photo and Mylar 
(Not to Scale) 



Sample : U/A 



These were winter wheat fields but are currently 
being plowed under due to the effects of drought. 
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STANDARD CROP KEY 


KEY 

W 

SW 

F 

G 

H 

A 

P 

C 

SF 

SU 

SG 

SR 

SY 

SB 

FX 

T 

R 

B 

X 

BN 

0 

(Crop)/H 

(Crop)/A 


CROP TYPE 
Winter Wheat 
Spring Wheat 
Fallow 

Grass (not cut for hay and no fence) 

Hay (any visible signs of hay activities) 

A1 falfa 
Pasture 
Corn 

Safflower 
Sunfl ower 
Sudan grass 
Sorghum 
Soybeans 
Sugar beets 
Flax 
Trees 
Rye 

Barley 

Homestead - nonag, lakes, ponds, etc. 

Beans 

Oats 

Crop has been harvested 

Crop has been abandoned; footnote and explain 


1. If there are crops in segment for which there is no code, select 
an unused symbol and indicate its use on the evaluation form. 

2. Use standard key for all identification. 

3. Use ball point pen for all coding on mylar. 
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CROP STAGE KEY 


CROP STAGE KEY 

STAGE 

DESCRIPTION 

1.0 

Planted 

Seed was put in the ground. 

o 

• 

CM 

Emerged 

When one leaf per plant is visible. 

3.0 

Jointed 

Defined as when the first node of the 
stem is visible. 

4.0 

Heading 

Defined as the stage when the base of 
the rachis (or head) reached the same 
height as the ligule (or base of the 
shot leaf). 

5.0 

Soft Dough 

At this stage the crop is starting to 
turn color. The kernals can be easily 
deformed when pressed between the 
fingers, but no "milk* 1 or liquid 
should exude under such pressure. 

6.0 

Hard Dough 

The kernals readily part from the head 


The grain is f i rjn and though it may be 
dented by pressure of the thumbnail, 
it is not easily crushed. The charac- 
teristic color of the grain has become 
more distinct. The leavers are brown, 
dry, and snrunken. Wheat in this stage 
may be swathed in some areas. 

7.0 Harvested or Straw is brittle and dull yellow az 

Harvestable this stage. The grain (if not har- 
vested yet) is hard and breaks into 
fragments when crushed. 
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EVALUATION FORM 


Segment No.: County: State: 

Name: Date: 

Man-Hours Required to Complete Survey: - 

I. Based on your assessment of the development of wheat in the segment 
while completing the survey, what is the average wheat stage for the 
segment? See attached Crop Stage Key. Is the crop development this 
year in the segment normal, ahead, or behind as compared to previous 
years? Explain. Enter Crop Stage: 


II. Comnents, footnotes, and additional crop key used: 


III. Comments on the effects of drought and/or winterkil 1 : 


IV. Comments and recommendations for improving these procedures for future 
surveys: 
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APPENDIX E 


FORMS USED IN TAKING INVENTORIES 
OF LAC IE INTENSIVE TEST SITES 

The following forms are used to record ground observations in the 
LACIE ITS’s: 

Figure Form 

E-l Sample Ground Truth Inventory Form 

E-2 Sample Ground Truth Periodic Observation 

Form 

E-3 Sample form for Ground Truth Data Collection 

System Rainfall Measurements 

E-4 Sample Yield Form 


E-l 



M 

K) 


?! 

A Kl 


MHtMB / oBiBM ' ,N °-’ 

100 5PRJW, l, H6AT (GCfJj 
corf list attache 6 


ground truth inventory toiih 


// H 

OAtA RECURDEO OETW|E.I7 1 x / -jr)_ /Jg 


IOO lirnCY (GEN) 
20 1 Laif.cn 
in? VAI.UUAF.0 
20) BONANZA , 

204 UILNS''N 

205 THIMUS 

206 $HAV|iET 
20 7 PI Mill NE 
201) i)H/t$ 

20/ SI6P10E 
210 FCKGUS 


REFtRCNCfe 
OF HELD 

ACREAGE 

"Jk- 

J-Zdl- 

JJ— 

-SI- 

-21— 

S-J2- 

-35- 

Jir-fL- 

-JOl.- 

JJLJ- 

-JA- 

-AJ— 

Jj-1- 

J3.J- 

sa. 

_X-2. 

jjjj 

JLX- 

'jji- 

jJLJi- 

JM- 

-laJL- 

Will- 

J±±- 

-L'JJ- 

-A-i- 

J\J- 

213- 0- 

-US 

JJi-L- 

JJ3- 


MS 

32 Js- 

-its 

SIS- 


-JLC- 

-ACS 

J3/.S- 

JUJ 

-j-i-i SX— 

•.r i-Alk 


1 IS- 

J12.JL- 

JLSiL 

Jfuf— 

Sli- 

JSJl- 


A 

1 

CL 

o 

r) 


Jj 

i- 

A 

jj 

JZ 

Q- 

Jl 

j. 

A- 

JJ 

i 

1. 

A 

-L 

j- 

Jl 

J- 

1- 

A 

-J 

0- 

Jl 

-L 

3- 

A 

-L 

-j_ 

J 

-L 

A 

A 

JL 

'I- 

ji 

-L 

I- 

JL 

f. 


rr 






A 

-1 

A 

Jl 

J 

A 

JJ 

-JL 

■J~ 

** 

X 

A 

Jj 

j - 

A 

A 

A 

X 

A 

ji 

£L 

Jl 

-i 

A 

-1 

J. 

2- 


LAND USE CRLP COOES 


211 PARAGON 

212 ll! Ft T A 

213 CUN LUC 5 t 

214 hi SCIJNS IN 

215 HClTuh 
2J3 IHP/N4 
217 MORAVIAN 
200 LA 1 S I Ct N ) 

301 HJSSLLL 

302 BASIN 

303 MUSTY 

304 RLUNLY 
303 LODI 

LAND USE 
CROP CODE 


3UI. 

30 


n l SS 1 UN 

MM CD WHEAT (CtH) 
0 A| K HARD N&IUHfcRN 
scuuf 


x Ins 

II All HUN 

300 MISSION 
'.00 
AO 1 

402 

403 PAMKfcrt 

404 b AOL t 

405 APACHE 

406 TblJhPH 
40 7 SAIanTA 

400 ' ' 

409 


CCNrURK 
ill SUN 


IRRIGATED r£RT ILIIED 

Y n) (£) N 

v ^ <?; N 

Y C N) (T) N 

Y Hi) !'0 U 

v oP a) « 

Y tfi) (i) N 

Y Qt) Q N 

Y (ijO © N 

Y (n) (y) N 

Y l«) S) M 

Y (n) (7) 71 

y Qy SO m 

v SO (0 N 

Y (I}) (Y> N 

Y («) <$) N 

r-T!^- -crrir 

Y 


Ci) 

(n; 

(s) 

(S) 

CtP 

<■£. 

(>0 

so 

(h) 


410 K'.llStR 
Ml MCCALL 

412 OHlVIjA 

413 GAIMS 

414 NUuAINES 

415 ORUCMaHP 
4 Id rif'O 
417 UUP1 
410 BH1LG6R 

419 API HUE 

420 TASCUSA 

421 STURDY 

422 CCNCKO 

PLANTING DATE 
HOIITH DAY YEAR 


423 CAI’hCCK 

424 [AGLL SCOUT 

425 IPAPPIR 

426 CHLYLNNE 

427 WlNALIA 

5^3 Kra^SCS/PASTURI 
600 OTHI II CKUPS IGC 
40 1 RAPtSECD 
602 RYE 
60S MUSTARD 

604 FLAX (GEN) 

605 NORALTA FLAX 


606 R( OHOlID PLAX 

607 CUM; 

610 BUCKWHCAT 
111 SUNTLOhCRS 
612' DRY BEANS 

613 LLNTlLS 

614 DRY PtAS 
TC. 615 Sugar af.ETS 

616 OP A I II SURGHUH 

617 SOYCIANS 
610 C01 TON 

700 SUMMLIt FALLOW _ 

uoo nijn-alriculTurE 

900 UNKNOWN CROPS 
COMMENTS 


jQ i JL 

- 3 / 2 1 

- Wt Jf- 
ja / J- 
-If / -10 

- 10/ Ji 

- /2 / J ■&. 
-/O < J/- 
-/Q / J/- 
-/a / J~- 
_ /f / S- 

- if / jf- 
_'V / J- 
jg / 

JL / J- 

-T 
Jjl / J<L 
-/a / j± 

_/J? / Jf- 

- 'J / _/'_ 
-If / Jl 
-/!'/ jo. 

-h 0 / Ji. 

- a >/ -/- 

-/£ / Ji 


1 C 

_/JL 

u\ 
20 - 
21: 
u: 
za 
is 
jo : 
2 i i 

is. 

2 iZ 

ZM 

Jil 

20Z 

Ztl 
23 1 

JX 

JX 

js 

JS 

jj- 

zsz 



JjLZ-tjO-JiL-Jf- 


lafJJ-ilzJ-'L-j.l—j.-’— £-6 — 2^_u_j — 

1<LL >.S:-u-0LLJi2 L-Sf 

Jj OL'lJa.A/LJZ &-L/X L^—J/SjZ- 






ii’r. 


JUJc-if— 

.Z-iijJJL- 


j!/-- 

— Ct— -fi—jf i—Jj/fZ . . 


L>-' 

3 z:jj. « 



Jc-xUi-c Li. 

faJ'Ljit AL-JJ-' Oi-^- 

iuuj. 


Figurel E-l Sample Ground Truth Inventory Form 
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TEST SITE I 25 (SHELBY, INDIANA) 
□ USER VAT ION 01 


LAND USE CODES 

100-SPRING WHEAT 
200 BARLEY 
300-0 ATS 

400 WINTFP WHEAT 
5G0-GP AS SE S /PASTURE 
600-DTHhR CROPS 
60 L-R API: SEED 
602-RU 
60 A EL AX 
607-CORN 
6L7-SDY0EANS 


GROWTH STAGES 

S^-f'LAN^ErD^NO^E MERGENCE 

03- EMERGENCE 

04- T1 LLER l NG § PR E BOOT , 

PREDUO 

05- nnono or nuonco 

06- UEG1 NNI ND TO HEAD 

OR r LOWER 

07- FULi Y HI ADED OR 

ELOWEJM 0 


GROUND TRUTH PERIODIC OBSERVATION FURM 

LANDSAT PASS DATE 
OBSERVATION DATE 
RAINFALL SINLE LAST 
FJCLO OPERATIONS 


GROUND 
COVER U> 

1- 0- 19 

2- 20- 39 
3” 40- 59 

4- 60- 79 

5- UO-lOO 


SURFACE HOISTURC 
CONDIT IONS 

1- OHY 

2- DAMP 
3 WET 

4-STAND 1 NG WATER 


WEED GROWTH 


1- NFGL I G IDLE 

2- SLIGHT 


MONTH 0\Y 

' J7_ / 76 

observation f _Az 


.IN. 


01- BAftE 

02 - 0 “ 


GROUND 

__ RARE OISKLO/CULTIVATED 

03- DARE PLOWED 

04- flAR ESECDED 

05- STANDING STUBBLL 
“ l)t SKED/CUL TI VATED 

PLOWED 
SEEDED 


1)6 STUDIIL- 
07-StUBHLi: 

OB- STUBBLE 
09 BURNED 

10- GKjULO 

1 1- WINDROWED OR SWATHED 


growth/yield 

DCTRACTANTS 

01- SAUNI TY 

02- 1 NSEC T S 

03- 01 SEASE 
04 DROUGHT 
05 -MU I STUFF 
Oi>-Wl NO 

07 HAIL 
Oil -FROST 
09 BIROS 


STAND 

QUALITY 

1 * POOR 

2- BEl DW 

AVERAGE 

3 - AVER AGE 

4- AUOVh 

AVERAGE 
5 EXCELLENT 
6 -ones .NUT 
APPLY 


6 1 0-COTTON 

09-RIPE MATURE 

3-MODE RAT fc 

12-MOWFD Oft COMBINED 

10-POT HOLFS 

13 ‘WINTERKILL 

700 SUMMER FAl LOW 
9QQ-UNKNQWN CROPS 

10- HARVE STf D 

11- DOCS NOT APPLY 

4 HEAVY »■ 

13 STACKED OR BALED 
14-01 1IC R 

11- UNFVrN STAND 

1 2- WLEOS 

14 - LODGING 

15- OTHER , 


FIELD 

NO. 

ACREAGE 

LAND USE 
CODE 

GROWTH 

STAGE 

GROUND 

COVER 

PI ANT 
HEIGHT 

SURFACE 

MOISTURE 

WEED 

GROWTH 

F If LO 

OPERATIONS 

hi 

D 

iDWTH/Yl f LD 
UR ACTANTS 

STAND DUALITY 
RATING 

CDMM 

-NTS 




(CIRCLE 1 

CIRCLE 

{INCHES) 

(CIRCLE 

(CIRCLE 

[CIRCLE 

(CIRCLE 

ONE ) 

{CIRCLE ONE) 

' 






ONE) 

$ 

ONE) 


ONE) 

CD 2 

3 4 

nriH) 

6)2 
} 4 
1 


one ) 








i 


iai 

Jlk 

Aoo 

01 

05 

09 

02 

06 

ID 

03 

07 

11 

VS 

Dill! 

01 

06 

n 

02 

07 

12 

03 

OB 

13 

04 05 

<?v u 

8 i 

n 

D 2 

07 

12 

03 

OB 

13 

04 

09 

14 

05 

10 

15 

Wo 

Y 

1 

H 

% 

315. 

All.. 

® 8 i 

03 

07 

04 

OQ 

®*V 

DID 


®> 2 4 

01 

06 

*P 

03 

OU 

04 05 
09 10 

01 

06 

02 

07 

03 

OB 

04 

09 

05 

ID 

W© 

1 Y 

N 




09 

10 

11 





ii 

12 

13 

14 

U 

12 

13 

14 

15 

] 


£2. 

till 

.LoO. 

84® 

8 ? 

04 

OO 


CUD 


®3 2 , 

01 

06 

02 

07 

03 

Oti 

1 $ 10 

84 

8 ? 

03 

OB 

04 

09 

05 

10 

w® 

Y 

N 


09 

10 

u 



Q 2 * 

<^4 

11 

12 

13 

fe 

11 

li! 

li 

14 

15 



fits. 

<311 

Joo 

01 

05 

02 

06 

03 

07 


Vd 

DID 

01 

06 

02 

07 

03 

OB 

01 

06 

02 

07 

03 

OH 

04 

09 

05 

10 


' Y 

N 


09 

10 

LI 



®, 2 , 

li 

12 
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Figure E-2. Sample Ground Truth Periodic Observation Form. 
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Figure E-3.— Sample Yield Form. 
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Figure E-4.— Sample form for Ground Truth Data Collection System 


_ / 76 

/ _ 

_ /76 

IN. 

_ /76 

/ _ 

_ /76 

IN. 

- /lb 

/ _ 

_ /76 

IN. 

_ / 76 

/ _ 

_ /76 

IN. 

_ /lb 

_ _ / _ 

_ /lb 

IN. 

_ F76 

/ _ 

_ t Ib 

IN. 

_ /lb 

/ _ 

_ /76 

IN. 

_ nb/ 

k dt ^ib 

IN. 

St 76 

! _ 

_ /76 

IN. 

_ V76 

/ _ 

_ /lb 

1 IN. 

_ /76 

/ _ 

_ /76 

IN. 

_ Hb 

/ _ 

_ /76 

IN. 

_ nb 

/ _ 

_ /lb 

IN. 

_ nb 

/ _ 

_ /76 

IN. 

_ /76 

/ „ 

_ /76 

IN. 

_ / 76 

/ _ 

_ /76 

IN. 

_ /76 

/ _ 

_ /76 

IN. 

_ /76 

/ _ 

_ /lb 

IN. 

_ /lb 

/ _ 

_ /76 

IN. 

-lib 

/ _ 

_ /7G 

IN. 

. _ /lb 

/ _ 

_ /lb 

IN. 

. _ /76 

/ _ 

_ / 76 

IN. 

. _ /lb 

/ _ 

_ /lb 

1 N. 

System 

Rainfall 

Measureim 


§g 

11 

£ > 

9* 



APPENDIX F 


CROP GROWTH STAGE REPORTING ■ FORMS 



GROWTH STAGE DATES FOR 10* DEVELOPMENT 
MONTH AND DAY OF MONTH 


Check one: 

Q Spring Wheat 
□ Winter Wheat 
Crop Year 


ITS SEG. NO. 

CRD # 

PLANTING 

DATE 1 

EMERGENCE 

DATE 1 

JOINTING 

DATE 2 

HEADING 

DATE 2 

SOFT 

DOUGH 

DATE 3 

RIPE 

DATE 1 * 

HARVEST 

DATE 5 

- 

\ 

\ 

i 

i 









1 Date at which 10* of fields In CRD were planted jor emerged, respectively. 

1 Date at which 10% of fields in CRD had begun to joint or head, respectively. 

1 Date at which 10% of fields in CRD had begun to enter soft dough stage (turning color to greenish-yellow to yellow). 

** Oate at which 10% of fields in CRD are ripe (hard dough) -stage or when they were swathed, (Indicated swathed If 

appl icable) . 

1 Date at which 10% of fields in CRD have been harvested either as standing grain or out of swath. 


Check one: 


[3 Spring Wheat 

GROWTH STAGE DATES FOR 50% DEVELOPMENT □ Winter Wheat 



1 Date at which 50% of fields in CRD were planted or emerged, respectively. 

2 Date at which 50% of fields in CRD had begun to joint or head, respectively. 

* Date at which 50% of fields in CRD had begun to enter soft dough stage (turning color to greenish-yellow to yellow). 

4 Date at which 50% of fields in CRD are ripe (hard dough) stage or when'they were’ swathed, (Indicated swathed if 

applicable}. 

. 1 Date at which 50% of fields in CRD have been harvested either as standing grain or out of swath. 



GROWTH STAGE DATES FOR 90% DEVELOPMENT 
MONTH AND DAT OF MONT H 


Check one: 

0 Spring Wheat 
0 Winter Wheat 
Crop Year^ 


i 

u> 


ITS SEG. NO. 

CRD § 

PLANTING 

DATE 1 

EMERGENCE 

DATE 1 

JOINTING 

DATE 2 

HEADING 

DATE 2 

SOFT 

DOUGH 

DATE 3 

RIPE 

DATE 1 * 

HARVEST 

DATE 5 

¥ 


| 

| 



• 





1 Date at which 90 % of fields In CRD were planted or emerged, respectively. 

2 Date at which 90% of fields In CRD had begun to joint or head, respectively. 

* 

1 Date at which 90% of fields in CRD had begun to enter soft dough stage (turning color'to greenish-yellow to yellow), 

" Date at which 90% of fields in CRD are ripe (hard dough) stage or when they were swathed, (indicated swathed if 

appl icoble) . 

3 Date at which 90% of fields In CRD have been harvested either as standing grain or out of swath. 
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